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Introduction 


Audit  Objective 


The  Legislative  Audit  Committee  requested  a  performance  audit  of 
the  Montana  Vocational  Rehabilitation  Program  (MVR)  administered 
by  the  Department  of  Health  and  Human  Services  (DPHHS).  We 
reviewed  two  major  areas  in  separate  audits  -  program  effectiveness 
and  efficiency  and  purchasing  procedures  for  clients'  goods  and 
services.  We  combined  the  results  in  one  report. 

The  audits'  objectives  were  to  examine  various  MVR  management 
and  purchasing  controls  to  determine  program  success  and/or 
effeaiveness.  To  make  these  determinations  we  addressed  the 
following  questions: 

1 .  What  types  of  individuals  are  served  by  the  MVR  Program? 

2.  What  types  of  goods  and  services  were  provided  to  MVR  clients? 

3.  How  does  the  department  ensure  vendors  provide  agreed  upon 
goods  and  services  and  purchasing  procedures  are  followed? 

4.  Are  vocational  evaluation  tests  purchased  for  MVR  clients 
providing  counselors  with  information  needed  to  make 
reasonable  decisions  concerning  a  client's  vocational  aptitude? 

5.  How  much  does  DPHHS  expend  on  MVR  goods  and  services? 

6.  What  are  the  outcomes  of  the  MVR  Program? 

7.  Are  organizational  controls  consistent  with  the  intent  of  the 
MVR  program? 

8.  How  does  the  MVR  Program  monitor  its  success  and/or 
effectiveness? 

We  answered  these  questions  by  reviewing  client  files;  budget 
reports;  expenditure,  funding  source  and  accrual  data;  policies; 
procedures;  goals  and  objectives;  federal  regulations,  and 
interviewing  department  staff. 
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Program  Eligibility 
Criteria 


The  mission  of  the  MVR  Program  is  "Promoting  work  and 
independence  for  Montanans  with  disabilities."  In  order  to  achieve 
this  mission,  MVR  provides  goods  and  services  to  people  of  an 
employable  age  with  physical  or  mental  disabilities  to  help  them 
secure  and  maintain  employment.  The  federal  government 
established  16  years  old  as  an  employable  age. 


An  individual  is  eligible  for  vocational  rehabilitation  services  if  they 
meet  two  specific  criteria:  1)  an  individual  must  require  vocational 
rehabilitation  services  to  prepare  for,  enter,  engage  in,  or  retain 
gainftil  employment;  and  2)  the  individual  must  either: 

A.  Have  a  physical  or  mental  impairment  which  constitutes  or 
results  in  a  substantial  impediment  to  employment,  and  can 
benefit  in  terms  of  an  employment  outcome  from  vocational 
services:  OR 

B.  Be  blind  or  have  a  disability  and  be  eligible  to  receive  Supple- 
mental Security  Income  (SSI)  or  Social  Security  Disability 
Insurance  (SSDI).  (These  individuals  are  also  considered  to  have 
severe  physical  or  mental  impairments  which  seriously  limit  one 
or  more  functional  capabilities  in  terms  of  employment 
outcomes. 

There  are  no  minimiim  or  maximum  income  requirements  to  be 
eligible. 


Goods  and  Services 
Provided 


MVR  provides  counseling  and  guidance,  vocational  evaluation,  and 
vocational  placement  services  without  cost  to  an  eligible  person. 
MVR  requires  rehabilitation  services  be  provided  at  public  cost  only 
to  clients  found  to  require  financial  assistance.  Vocational 
rehabUitation  goods  and  services  purchased  by  MVR  include  medical 
diagnosis,  vocational  evaluation,  educational  training,  occupational 
tools  and  equipment,  physical  and  mental  restoration,  personal 
assistance,  transportation,  and  post  employment  services. 


Services  are  delivered  by  71 .5  FTE  located  in  ten  offices  across  the 
state  and  the  central  office  in  Helena.  The  program  is  funded  by 
state  and  federal  funds.  State  funds  consist  of  both  General  Fund 
and  state  special  revenue.  Over  70  percent  of  fiscal  year  1994-95 
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expenditures  were  for  client  benefits  and  claims.  Total  expenditures 
for  fiscal  year  1994-95  were  approximately  $10  million. 


Individualized  Written 
Reliabilitation  Program 
Process 


Once  an  applicant  is  determined  eligible  for  vocational  rehabilitation 
services,  an  Individualized  Written  Rehabilitation  Program  (IWRP) 
is  developed  by  the  vocational  rehabilitation  counselors  in 
collaboration  with  the  client.  The  IWRPs  essentially  outline 
rehabilitation  objectives  relating  to  attainment  of  the  employment 
goal,  and  contain  a  list  of  specific  vocational  rehabilitation  goods 
and  services  to  be  provided.  Both  the  counselor  and  the  individual 
receiving  services  from  MVR  use  the  IWRP  because  it  outlines  the 
goals  and  objectives  necessary  to  lead  to  an  employment  outcome. 


A  client's  case  can  be  closed  at  anytime  during  the  vocational 
rehabilitation  process.  For  example,  a  case  file  can  be  closed  when 
it  is  determined  the  individual  is  ineligible,  refuses  services,  does  not 
cooperate,  is  no  longer  available  for  services,  or  has  achieved  a 
successful  outcome. 


Program  Efficiency  and 
Effiectiveness  Could  be 
Increased 


During  the  audit  we  reviewed  the  processes  counselors  follow  for 
determining  services  needed,  providing  services,  and  measuring 
success.  We  believe  some  of  these  processes  could  be  more  efficient 
and  effective. 


Policies  Need  to  be 
Clarified  or  Written 


Prior  to  the  1992  amendments  to  the  federal  Rehabilitation  Aa  of 
1973,  MVR  had  a  policy  manual  which  was  overly  detailed.  As  a 
result  of  the  1992  amendments,  the  MVR  stafi"  rewrote  the  policy 
manual  with  emphasis  on  empowering  counselors  to  allow  them  to 
make  decisions  on  their  own. 


We  believe  management  made  the  new  policy  manual  too  general  in 
some  areas.  We  recommend  MVR  Program  management  clarify 
some  existing  policies  and  develop  additional  policies  and 
procedures  related  to  the  IWRP  and  purchasing  to  increase  the 
program's  efficiency  and  effectiveness.  Hie  department  began 
implementing  the  recommendations  by  drafting  and  implementing  a 
number  of  policies  during  the  audit.  The  following  identifies  issues 
discussed  in  the  report  and  resulting  program  action: 
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Review  Closed  Cases 


Once  policies  are  in  place  and  training  is  complete,  central  and 
regional  office  management  should  continue  to  periodically  review 
counselors'  work  to  ensure  policies  are  followed.  A  number  of 
counselor  files  are  reviewed  annually,  but  most  cases  are  open 
(active)  at  the  time  of  review.  A  review  of  more  closed  files  would 
allow  management  to  determine  how  a  counselor  is  dealing  with 
every  aspect  of  the  program,  from  application  to  closure  to  post 
employment. 

MVR  drafted  a  policy  so  closed  cases  will  represent  60  percent  of  the 
files  reviewed  annually  ft)r  each  counselor. 


Organizational  Controls 
Need  to  be  Reviewed 


To  increase  effectiveness,  MVR  management  should  review 
organizational  controls  which  include  goals  and  objectives, 
management  information,  and  performance  measures.  Program 
goals  should  relate  to  the  mission  and  purpose  of  the  program. 
Objectives  should  relate  to  the  goals  and  should  describe  specific  and 
measurable  targets,  results,  or  outcomes.  Management  information 
should  provide  a  mechanism  to  gather  pertinent  information  for  tise 
in  determining  whether  the  program's  goals  and  objectives  are  met. 
Management  information  also  allows  program  staff  to  develop 
performance  measures  which  help  determine  program  success. 


MVR  management  indicated  their  Strategic  Plan  (written  every  year 
to  supplement  the  State  Plan  written  every  three  years)  will 
contribute  to  the  development  of  more  tangible  measurable  goals  and 
objectives.  They  believe  the  new  computer  system  will  help  them 
address  their  management  information  needs. 


Summary 


Audit  work  showed  the  MVR  Program  is  meeting  its  mission  of 
promoting  work  and  independence  for  Montanans  with  disabilities. 
We  reviewed  118  eligible  client  files.  Thirty-two  (27  percent)  were 
closed  after  the  clients  were  successfully  employed  for  60  days  and 
another  eight  individuals  received  post-employment  services  from 
MVR  and  were  employed  for  the  60-day  requirement.  We  reviewed 
employment  status  of  24  of  the  32  successfully  closed  clients  and 
found  20  of  the  24  continued  to  be  employed  as  of  the  first  quarter 
of  1996.  In  addition,  five  of  the  eight  clients  on  post  employment 
continued  to  be  employed  as  of  the  first  quarter  of  1996. 
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The  department  began  implementing  the  majority  of  the 
recommendations  during  the  audit.  We  believe  the  program's 
efficiency  and  effectiveness  will  increase  when  the  recommendations 
are  fiilly  implemented.  By  increasing  the  efficiency  and 
effectiveness,  the  program  could  increase  the  number  of  clients 
categorized  as  "closed  rehabilitated,  successfully  employed  for  at 
least  60  days." 
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Introduction 


The  Legislative  Audit  Committee  requested  a  performance  audit  of 
the  Montana  Vocational  Rehabilitation  (MVR)  Program  administered 
by  the  Department  of  Public  Health  and  Human  Services  (DPHHS). 
During  preliminary  planning  we  identified  two  major  areas  to 
review,  program  effectiveness  and  purchasing  procedures  for  clients' 
goods  and  services.  We  conducted  audits  of  both  areas  and 
combined  the  results  in  one  report. 


Audit  Objectives 


The  audits'  objectives  were  to  examine  various  MVR  controls  to 
determine  program  success  and/or  effeaiveness  and  if  purchasing 
procedures  ensure  MVR  clients  receive  goods  and  services 
purchased  for  them.  To  make  these  determinations,  we  addressed 
the  following  questions: 


1 .  What  types  of  individuals  are  served  by  the  MVR  Program? 

2.  What  types  of  goods  and  services  were  provided  to  MVR 
clients  during  fiscal  year  1994-95? 

3.  How  does  the  department  ensure  vendors  provide  agreed  upon 
goods  and  services  and  purchasing  procedures  are  followed? 

4.  Are  vocational  evaluation  tests  purchased  for  MVR  clients 
providing  counselors  with  information  needed  to  make 
reasonable  decisions  concerning  a  client's  vocational  aptitude? 

5.  How  much  did  DPHHS  expend  on  MVR  goods  and  services 
for  fiscal  year  1994-95? 

6.  What  are  the  outcomes  of  the  MVR  Program? 

7.  Are  organization  controls  consistent  with  the  intent  of  the 
MVR  Program? 

8.  How  does  the  MVR  Program  monitor  its  success  and/or 
effeaiveness? 
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Audit  Scope  and 
Methodology 


The  audits  were  conducted  in  accordance  with  government  auditing 
standards  for  performance  audits.  We  reviewed  the  1992  amend- 
ments to  the  federal  Rehabilitation  Act  of  1973  establishing  the 
vocational  rehabilitation  program.  We  also  reviewed  proposed 
federal  regulations  implementing  the  1992  amendments.  A  copy  of 
the  most  recent  State  Plan  was  reviewed.  Department  policies, 
procedures,  goals,  and  objectives  were  obtained  and  reviewed. 
These  reviews  provided  us  with  information  and  criteria  to  analyze 
the  program.  We  also  solicited  comments  from  the  Vocational 
Rehabilitation  Advisory  Council. 


We  examined  a  random  sample  of  150  client  files  closed  in  fiscal 
year  1994-95.  At  least  34  files  were  reviewed  in  each  of  the  four 
regional  offices.  During  the  file  review  we  gathered  information 
pertaining  to: 


Client  demographics. 

Activities  occurring  from  application  submission  to  case 

closure. 

Client  financial  summary  information. 

Goods  and  services  provided  and/or  purchased  for  each  client. 

Contributions  by  MVR  clients  and  other  state  or  federal 

programs  toward  cost  of  goods  and  services  provided  by 

DPHHS. 

Compliance  with  department  policies. 


Client  information  was  gathered  during  the  file  review  and  compared 
to  information  on  DPHHS's  MVR  management  information  system. 
This  procedure  was  conducted  to  determine  if  the  information  on  the 
system  was  accurate. 

Budget  report,  expenditure,  funding  source  and  accrual  data  for 
fiscal  years  1992-93  through  1994-95  was  gathered  to  determine 
DPHHS  expenditures  for  MVR  personal  services,  operating 
expenses,  equipment,  and  goods  and  services  for  clients.  We 
reviewed  management  information  DPHHS  compiles  to  identify 
types  of  goods  and  services  provided  to  clients. 

We  interviewed  staff  in  the  central  and  regional  offices  to  gather 
information  pertaining  to  management  controls.  We  also 
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interviewed  MVR  staff  to  determine  if  vocational  evaluation  tests 
conduaed  by  accredited  facilities  provide  information  needed  to 
adequately  evaluate  clients. 

We  compared  information  on  financial  summaries  to  information 
from  the  Department  of  Labor  and  Industry,  DPHHS,  universities, 
colleges  and  vocational  schools,  and  State  Fund  records.  These 
comparisons  allowed  us  to  determine  if  clients  correctly  reported 
income  sources  and  amount  of  income  on  financial  summaries. 

For  specific  closed  cases,  we  accessed  the  Unemployment  Insurance 
system  to  determine  if  people  were  still  employed  and  to  help 
determine  program  success. 

The  department  sends  surveys  to  clients  to  obtain  the  clients' 
perception  of  the  program.  We  reviewed  the  surveys  to  analyze  how 
they  are  used  in  determining  program  success. 

Compliance  Although  we  did  not  specifically  review  compliance  with  federal  or 

state  regulations,  we  tested  timeliness  of  determining  eligibility  and 
Individualized  Written  Rehabilitation  Program  (IWRP)  development. 
The  Montana  Vocational  Rehabilitation  Program  underwent  a  State 
Plan  Assurance  Review  (SPAR)  in  March  1993.  SPAR  is  a  federal 
review  designed  to  evaluate  selected  assurances  stated  in  the  State 
Plan.  Tliere  were  no  significant  findings  in  the  SPAR  which  related 
to  our  audits. 

In  May  1996,  the  MVR  Program  underwent  a  federal 
Comprehensive  Monitoring  and  Technical  Assisunce  Review  of 
Section  107  Requirements  of  the  Rehabilitation  Act  of  1973,  as 
amended.  Section  107  requires  the  federal  Rehabilitation  Services 
Administration  to  conduct  aimual  reviews  and  periodic  on-site 
monitoring  reviews  of  Title  I  Vocational  Rehabilitation  service 
programs  to  determine  whether  the  state  complies  substantially  with 
provisions  of  its  State  Plan.  Areas  of  review  included: 

»•      Achievement  of  employment  outcomes. 
»•      Requirements  for  informed  choice. 
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►  Procedures  to  identify  and  service  individuals  with  disabilities 
from  minority  backgrounds. 

»^      Due  process  hearings. 
►•      The  strategic  plan. 

►  Consumer  satisfaction  surveys  and  other  information  provided 
by  the  State  Rehabilitation  Advisory  Council. 

►  Order  of  selection  policies. 

►  Timeliness  of  reports  submitted  by  DPHHS. 

►•      Financial  management  which  focused  on  match,  carryover,  set 
aside,  reallotment,  and  maintenance  of  effort  requirements. 

The  federal  review  of  103  client  files  focused  on: 

Employment  outcomes  -  if  services  provided  were  necessary,  if 
MVR  services  contributed  significantly  to  the  employment  outcome, 
and  if  the  consumer  agreed  with  the  closure. 

Individual  Written  Rehabilitation  Programs  -  were  jointly  developed 
with  and  agreed  to  by  the  client,  goals,  objectives  and  services  were 
based  on  client  capabilities,  and  comparable  benefits  were  con- 
sidered and  utilized  when  available. 

Eligibility  -  eligibUity  determination  was  made  within  60  days,  there 
was  documentation  of  an  existing  physical  or  mental  impairment 
which  results  in  a  substantial  handicap  to  employment,  the  client  re- 
quired MVR  services  to  prepare  for,  enter,  engage  in  or  retain  gain- 
fiil  employment,  and  comprehensive  assessments  were  completed. 

Case  closure  -  client  was  notified  of  case  closure,  the  Client 
Assistance  Program,  and  the  appeals  process. 

Our  work  complemented  the  federal  review  in  two  areas  —  timeliness 
of  eligibility  determination,  and  establishing  goals,  objectives,  and 
services  based  on  a  client's  capabilities.  In  both  cases  the  federal 
review  findings  and  our  findings  are  similar.  These  issues  are 
discussed  in  detail  in  Chapter  IV. 
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Data  Limitations 


DPHHS  maintains  a  computer  system  to  gather  client  data.  When 
we  compared  information  in  the  files  to  information  on  the  computer 
system  we  found  inconsistencies  in  some  areas.  Some  problems 
were  due  to  the  inflexibility  of  the  computer  system.  For  example, 
the  computer  would  not  accept  dates  on  the  input  forms  which  were 
prior  to  the  date  the  information  was  actually  entered  onto  the 
system,  so  actual  application  and  closure  dates  could  not  be  entered. 
Also,  if  a  case  was  closed  before  all  the  bills  were  paid,  the  case  had 
to  be  reopened  under  another  status. 


We  also  found  differences  between  information  in  the  counselor 
notes  and  what  the  counselor  wrote  on  the  input  form.  We  found 
some  categories  on  the  input  form  were  completed  incorrectly  or  not 
always  completed. 

Because  of  problems  found  concerning  accuracy  of  some  information 
on  the  computer  system,  we  limited  the  use  of  DPHHS's  computer 
information  in  this  report.  The  numbers  identified  in  the  report  are 
based  on  our  file  review. 

A  new  computer  system  was  installed  in  July  1996,  after  we 
completed  the  majority  of  our  field  work.  We  sent  a  memorandum 
to  the  department  prior  to  implementation  of  the  new  system 
discussing  the  inconsistencies  we  found  in  the  old  system. 
Department  staff  indicated  many  of  the  problems  discussed  were 
addressed  with  the  new  system. 


Management 
Memorandum 


During  the  course  of  the  audit  we  presented  recommendations  to 
program  management  which,  if  implemented,  we  believe  could 
result  in  improvements  in  program  efficiency  and  effeaiveness. 
These  memorandums  allowed  DPHHS  to  be  aware  of  our  audit 
issues  prior  to  the  end  of  audit  work.  Department  staff  agreed  with 
all  reconmiendations  discussed  in  the  memorandum. 


We  also  sent  a  management  memorandum  to  DPHHS  containing 
recommendations  which  were  minor  relative  to  other  issues 
addressed  in  the  report.  The  memorandum  recommended: 
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►■      Establishing  policy  and  procedures  for  assignment  of  case  files 
in  the  event  a  counselor  is  on  vacation,  on  extended  leave  of 
absence,  or  there  is  a  position  vacancy.  Written  procedures 
would  help  ensure  supervisors  are  aware  of  the  need  to  assign 
cases  when  counselors  are  absent. 

►      Developing  policies  and  a  system  to  help  counselors  identify 
files  that  should  be  closed.  Written  policies  and  a  formalized 
system  would  help  ensure  files  are  closed  in  a  timely  manner. 

•■      Developing  policies  and  procedures  to  help  counselors 
consistently  categorize  client  files  at  closure. 

►•      Updating  the  policy  and  procedure  manual  to  identify  and 
define  policies,  current  purchasing  limits,  purchasing  forms, 
and  documentation  requirements. 

MVR  management  indicated  new  policies  addressing  the  above 
recommendations  are  in  draft  or  final  status. 


Report  Organization  Chapter  n  provides  information  about  the  Montana  Vocational 

Rehabilitation  Program's  history  and  answers  our  first  audit 
objective  concerning  the  types  of  individuals  the  program  serves. 
Chapter  III  provides  a  summary  of  program  efficiency  and 
effeaiveness.  Chapter  IV  discusses  the  process  used  to  serve  clients 
from  application  through  case  closure.  Purchases  of  goods  and 
services  are  described  in  Chapter  V.  Chapter  VI  summarizes  our 
findings  on  organizational  controls  and  program  success  and 
effectiveness. 
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Introduction 


Program  History 


The  Montana  Vocational  Rehabilitation  (MVR)  Program  is  located 
within  the  Disability  Services  Division  in  the  Department  of  Public 
Health  and  Human  Services  (DPHHS).  MVR  provides  services  to 
persons  of  employable  age,  with  physical  or  mental  disabilities,  and 
helps  restore  them  to  gainful  employment. 

In  1947,  the  Montana  Legislature  created  the  Division  of  Vocational 
Rehabilitation  to  administer  the  Vocational  Rehabilitation  Program 
established  under  the  federal  Social  Security  Act.  The  division  was 
responsible  for:  1)  providing  rehabilitation  services  to  the  state's 
physically  and  mentally  handicapped;  and  2)  performing  disability 
determinations  for  Social  Security  disability  benefits  and 
supplemental  income  payments. 

By  1965,  federal  amendments  eliminated  economic  need  as  a 
prerequisite  for  vocational  rehabilitation  (VR)  services.  If  desired, 
states  could  require  economic  need  tests  for  some  services  purchased 
after  the  person  is  determined  eligible  for  vocational  rehabilitation. 
When  economic  need  tests  are  conducted,  VR  counselors  review 
client  financial  information  to  determine  if  the  client  has  the 
financial  resources  to  help  pay  for  services  VR  is  purchasing.  An 
example  of  a  service  in  which  a  client  would  participate  would  be 
tuition  and  books  for  a  college  education.  Montana  chose  to  require 
an  economic  needs  test. 

The  most  recent  change  to  the  Act  is  the  1992  Rehabilitation  Act 
Amendments.  Tliis  change  made  adjustments  to  the  principles, 
purpose,  process,  and  outcomes  of  VR  programs.  The  1992 
amendments  emphasize  the  importance  of  appropriate  services  and 
support  to  people  with  disabilities  to  achieve  employment  outcomes. 
These  amendments  require: 

-      Assisting  an  individual  to  make  informed  choices  concerning 
potential  employment  outcomes. 

••      Development  of  an  Individual  Written  Rehabilitation  Program 
(IWRP)  with  the  full  participation  of  the  person  with  the 
disability. 
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State  Plan  and  State 
Advisory  Council 
Required 


►  Need  to  match  what  is  outlined  in  the  IWRP  with  appropriate 
services  and  support. 

►  Cooperative  working  relationships  with  other  agencies  and 
programs  including  local  education  authorities  to  coordinate 
transition  services. 

►■      Emphasizing  quality  of  services  provided  to  persons  with  a 
disability. 

►•  The  eligibility  decision  must  focus  on  use  of  existing  data,  and 
eligibility  must  be  made  within  60  days  unless  mutually  agreed 
upon  between  the  individual  and  the  VR  counselor. 

►  Establishing  the  eligibility  process  as  two  parts:  1)  determine  if 
the  person  has  a  disability  and  2)  determine  if  the  person 
requires  assistance  from  VR  to  achieve  an  employment 
outcome. 

The  Rehabilitation  Act  requires  each  designated  state  agency  to 
submit  a  State  Plan  every  three  years.  The  State  Plan  is  an 
agreement  between  the  federal  government  and  a  state  regarding 
program  administration  and  conduct.  In  the  Montana  State  Plan, 
DPHHS  commits  to  maintaining  written  policies  and  procedures,  and 
compliance  with  federal  requirements  and  mandates. 


The  Rehabilitation  Act  also  requires  states  to  have  an  advisory 
council  associated  with  their  rehabilitation  programs.  The 
Vocational  Rehabilitation  Advisory  Council  is  associated  with  the 
MVR  Program.  The  purpose  of  the  Advisory  Council  is  to  advise 
the  administrator  of  the  MVR  Program  concerning  policy  and 
program  issues,  delivery  of  services  to  consumers,  and  methods  for 
reaching  potential  consumers.  Council  functions  include  reviewing, 
analyzing,  and  advising  the  MVR  Program  relative  to  implementa- 
tion of  both  state  and  federal  statutes  which  affect  provisions  of 
rehabilitation  services  to  persons  with  disabilities.  The  council 
consists  of  17  people  who  either  have  disabilities,  represent  disabled 
groups,  represent  the  business  community,  or  have  an  interest  in 
disability  issues.  Council  members  are  appointed  by  the  Governor, 
serve  a  term  of  not  more  than  three  years,  and  may  not  serve  more 
than  two  consecutive  terms. 
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Advice  Sought  From 
Associated  Groups 


Program  EUgibility 
Criteria 


In  addition  to  the  Council,  MVR  staff  seek  advice  from  many 
different  groups  and  agencies.  MVR  works  closely  with  the  Client 
Assistance  Program,  Council  of  State  Administrators  of  Vocational 
Rehabilitation,  Job  Training  Partnership  Act  Private  Industry 
Councils,  Social  Security  district  offices,  and  the  Disability  Deter- 
mination Unit.  It  is  also  the  responsibility  of  regional  MVR  offices 
to  coordinate  with  local  community  rehabilitation  programs,  parents' 
groups,  school  district  personnel,  and  private  businesses. 

The  mission  of  the  MVR  Program  is  "Promoting  work  and 
independence  for  Montanans  with  disabilities."  In  order  to  achieve 
this  mission,  MVR  provides  goods  and  services  to  people  of  an 
employable  age  with  physical  or  mental  disabilities  to  help  them 
secure  and  maintain  employment  The  federal  government 
established  16  years  old  as  an  employable  age. 


An  individual  is  eligible  for  VR  services  if  they  meet  two  specific 
criteria:  1)  an  individual  must  require  vocational  rehabilitation 
services  to  prepare  for,  enter,  engage  in,  or  retain  gainful 
employment;  and  2)  the  individual  must  either: 

A.    Have  a  physical  or  mental  impairment  which  constitutes  or 
results  in  a  substantial  impediment  to  employment,  and  can 
benefit  in  terms  of  an  employment  outcome  from  vocational 
services;  OR 


B. 


Be  blind  or  have  a  disability  and  be  eligible  to  receive 
Supplemental  Security  Income  (SSI)  or  Social  Security 
DisabUity  Insurance  (SSDI).  (These  individuals  are  also 
considered  to  have  severe  physical  or  mental  impairments  which 
seriously  limit  one  or  more  functional  capacities  in  terms  of 
employment  outcomes.) 


TTiere  are  no  minimum  or  maximum  income  requirements  to  be 
eligible. 
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Goods  and  Services  MVR  provides  counseling  and  guidance,  vocational  evaluation,  and 

Provided  Vary  vocational  placement  services  without  cost  to  an  eligible  person.  In 

addition,  MVR  may  pay  the  cost  of  other  vocational  rehabilitation 

goods  and  services  for  an  eligible  person  in  financial  need. 

Vocational  rehabilitation  goods  and  services  purchased  by  MVR 

include: 

Medical  diagnosis  -  medical  and  psychological  records,  exams,  and 
evaluations. 

Vocational  evaluation  -  assessment  of  eligibility  and  vocational 
rehabilitation  needs. 

Rehabilitation  training  -  personal  and  vocational  adjustment  training 
including  job  coaching,  work  adjustment  training,  transitional 
employment  services,  and  orientation  and  mobility  services  for  the 
blind. 

Educational  training  -  college,  vocational  education  and  trade  school 
tuition,  books  and  supplies,  tutoring,  interpreter  services  for  the 
deaf,  and  reader  services  for  the  blind. 

Vocational  placement  -  job  finding,  employer  contact  and 
negotiation,  job  accommodation,  and  job  placement. 

Occupational  licenses  -  professional  licenses  such  as  electrical, 
plumbing,  nursing,  cosmetology,  and  teacher  exams  and  certificates. 

Occupational  tools  and  equipment  -  mechanic  tools,  cosmetology 
supplies,  computer  equipment  and  supplies,  and  other  equipment  and 
supplies  for  self-employment. 

Rehabilitation  technology  services  -  assistive  technology  devices 
including  telecommunication,  sensory,  and  other  technological  aids 
and  equipment. 

Phvsical  and  mental  restoration  -  corrective  surgery  or  therapeutic 
treatment,  prosthetic  and  orthotic  devices,  eye  exams  and  eye 
glasses,  ear  exams  and  hearing  aids,  dental  exams  and  dental  work. 
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transplantation  and  dialysis,  and  diagnosis  and  treatment  for  mental 
and  emotional  disorders. 

Personal  assistance  -  personal  care  attendant  services. 

Family  services  -  any  necessary  services  for  family  members  in 
support  of  the  client's  rehabilitation  including  day  care  and  respite 
care. 


Specialized  Services 
Provided  to  Individuals 
Not  Able  to  Enter 
Competitive  Employment 


Maintenance  -  additional  costs  incurred  while  participating  in 
rehabilitation  such  as  clothing,  hair  cuts,  living  expenses,  and  food 
and  lodging  if  travel  is  needed  for  evaluation. 

Transportation  -  gas,  vehicle  repairs  and  modification,  vehicle 
insurance,  parking  permits,  bus  passes,  and  moving  expenses. 

Post-emplovment  services  -  services  after  successful  job  placement 
for  the  purpose  of  maintaining  employment. 

For  those  clients  not  able  to  enter  competitive  employment,  MVR 
can  also  provide  independent  living,  supported  employment,  and 
extended  employment  services.  Independent  living  services  include 
making  housing  handicapped  accessible,  services  for  daily  living 
activities  (cooking,  cleaning,  shopping),  and  transportation  services. 
Supported  employment  is  for  individuals  with  severe  disabilities  who 
traditionally  have  been  unable  to  enter  competitive  employment 
because  of  an  impediment,  or  competitive  employment  was 
interrupted  as  a  result  of  a  severe  disability.  Supported  employment 
does  not  require  the  client  receive  minimum  wage.  However,  the 
person  must  be  paid  wages  on  a  basis  consistent  with  wages  paid  to 
non-disabled  workers  with  similar  job  functions.  Extended  services 
are  ongoing  support  services,  such  as  job  coaching,  which  is  needed 
to  support  and  maintain  a  person  with  a  severe  disability  receiving 
supported  employment. 


Page  11 


Chapter  II  -  Vocational  Rehabilitation  Background 


Services  Delivered  from 
Regional  Omces 


Services  are  delivered  to  clients  from  ten  offices  across  the  state. 
The  following  map  identifies  the  four  vocational  rehabilitation 
regions  in  Missoula,  Great  Falls,  Butte,  and  Billings,  and  the 
satellite  offices  in  Kalispell,  Havre,  Warm  Springs,  Bozeman, 
Helena,  and  Miles  City. 


Figure  1 
Montana  Vocational  Rehabilitation  Regions 


*  Region  Office 

•  Satellite  Office 


Source:  Compiled  by  the  Legislative  Audit  Division  from  MVR  records. 
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Program  Staffing 


Counselors  comprise  the  majority  of  fiiU-time  equivalents  (FTE)  in 
the  region  offices.  Each  area  has  a  Regional  Administrator 
overseeing  region  staff.  The  other  management  positions  are  located 
in  the  Helena  central  office.  Table  1  identifies  the  breakdown  of 
staff  in  the  Helena  central  office  and  the  four  regions. 


Table  1 

Vocational  Rehabilitation  FTE  Distribution 

(Fiscal  Year  1994-95) 


Office/Region 

Management* 

Counselors** 

Administrative 
Support 

Total 

Central  Office 

6.75 

0.00 

2.00 

8.75 

Great  Falls  Region 

1.00 

7.50 

4.50 

13.00 

Butte  Region 

1.00 

9.50 

5.00 

15.50 

Missoula  Region 

1.00 

9.50 

5.75 

16.25 

Billings  Region 

1.00 

11.50 

5.50 

18.00 

Total  MVR  Staff 

10.75 

38.00 

22.75 

71.50 

*          The  management  column  represents  all  program  officers  in  Helena  and  regional 

administrators. 
**         The  counselors  column  includes  counselor  supervisors  since  they  carry  a  client 

caseload. 

Source:    Compiled  by  the  Legislative  Audit  Division  from  MVR  records. 


Program  Funding  and 
Expenditures 


MVR  is  funded  by  state  and  federal  funds.  State  funds  consist  of 
both  General  Fund  and  state  special  revenue.  Table  2  identifies 
MVR  funding  sources  and  expenditures  for  state  fiscal  years  1992-93 
through  1994-95. 
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Table  2 

MVR  Program  Funding  Sources  and  Expenditures 

(Fiscal  Years  1992-93  through  1994-95) 


Fund  Sources 

1992-93 

1993-94 

1994-95 

Federal 

$7,547,960 

$8,177,204 

$  7,827,277 

State  General 

1,304,241 

1,817,486 

1,978,983 

State  Special 

645,875 

340,960 

200,266 

Total  Expenditures 

$9,498,076 

$10,335,650 

$10,006,526 

Source:    Compiled  by  the  Legislative  Audit  Division  from 
MVR  records. 


State  special  revenue  funds  declined  over  the  three  fiscal  years  due 
to  switching  funding  from  the  Workers'  Compensation  Industrial 
Accident  Rehabilitation  Account  to  state  General  Fund. 


Mjyority  of 

Expenditures  for  Client 
Benefits  and  Claims 


Over  70  percent  of  fiscal  year  1994-95  expenditures  were  for  client 
benefits  and  claims.  Table  3  shows  total  expenditures  for  the  MVR 
Program  for  fiscal  year  1994-95  by  area  of  expenditure. 
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Table  3 

MVR  Expenditures  bv  Category 

(Fiscal  Year  1994-95) 


Category 

Amount 

Percentage 
of  Total 

Personal  Services 
Operating  Expenses 
Equipment  and  Intangible  Assets 
Benefits  and  Claims 

$  2,244,694 

411,875 

27.520 

7.322,436 

22.4 
4.1 
0.3 

73.2 

Total  Expenditures: 

$10,006,525 

100.00 

Source:    Compiled  by  the  Legislative  Audit  Division  from 
Statewide  Budgeting  and  Accounting  System 
records. 


Federal  Section  1 10  funds,  which  require  a  21.3  percent  state  match, 
fiind  the  majority  of  program  administration  and  benefit  costs. 
Other  MVR  federal  funding  sources  include  Supported  Employment, 
Job  Training  Partnership  Act  (JTFA),  Independent  Living 
(90  percent  federal).  Migrant  and  Seasonal  Farm  Worker,  and 
MonTech  funds.  State  funding  includes  General  Fund  for  Extended 
Employment  (100  percent)  and  Independent  Living  (10  percent),  and 
state  special  revenue  for  Workers'  Compensation. 

The  funds  are  used  to  provide  different  types  of  goods  and  services 
to  clients.  The  following  lists  the  funds  and  their  uses: 

Section  1 10  -  vocational  evaluations,  counseling  and  guidance, 
training  and  education  support,  physical  and  mental  restoration,  job 
placement,  and  post-employment  services. 

Supported  Employment  -  time  limited  support  services  to  people 
with  severe  disabilities. 

JTPA  -  training  clients  over  age  22. 
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Extended  Employment  -  long-term  job  support  for  severely  disabled 
people  who  work  in  sheltered  or  supported  employment. 

Workers'  Compensation  -  Workers'  Compensation  cases  that  are 
also  MVR  clients. 

Independent  Living  -  helps  support  disabled  people  to  allow  them  to 
maintain  independent  living  situations. 

Migrant  and  Seasonal  Farm  Worker  -  provides  support  to  disabled 
migrant  and  seasonal  workers. 

MonTech  -  funding  for  a  contract  to  provide  information  and 
referral  services  as  well  as  purchase  technology-related  devices  for 
people  with  disabilities. 

Table  4  represents  expenditures  for  benefits  and  claims  for  clients  by 
type  of  funding  source  for  fiscal  year  1994-95. 


Table  4 
MVR  Benefits  and  Claims  Expenditures  for  Goods  and  Services  Purchased 

(Fiscal  Year  1994-95) 


Funding  Source 

Total 
Expenditures 

Percent 
of  Total 

Federal 
Funds 

State 
Funds 

Federal/State 
Percentage 

Section  110 

$4,918,366 

68 

$3,870,754 

$1,047,612 

78/21 

Supported  Employment 

297,421 

4 

297,421 

0 

100/0 

JTPA 

253,665 

3 

253,665 

0 

100/0 

Extended  Employment 

487,783 

7 

0 

487,783 

0/100 

Workers'  Compensation 

96,957 

1 

0 

96,957 

0/100 

Subtotal  Benefit  Expmditures 

6,054,192 

83 

4,421,840 

1,632,352 

73/27 

Other  funds:  MonTech, 
Independent  Living,  Migrant 

1,268,244 

17 

1,142,596 

125,648 

90/10 

Total  Benefits  Expenditures 

$7,322,436 

100 

5,564,436 

1,758,000 

76/24 

Source:    Compiled  by  the  Legislative  Audit  Division  from  Statewide  Budgeting  and  Accounting 
System  records. 
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The  other  funds,  such  as  MonTech,  Independent  Living,  and 
Migrant  and  Seasonal  Farm  Workers,  can  be  utilized  by  any  person 
with  disabilities,  not  just  MVR  clients. 


Types  of  Sampled 
Clients  Served  by 
Montana  Vocational 
Rehabilitation  Program 


We  compiled  client  demographics  from  a  random  sample  of  150 
client  files.  The  files  reviewed  were  cases  closed  in  fiscal  year  1994- 
95.  The  following  sections  provide  client  information  gathered 
during  the  file  review. 


Applicant  Gender  and  Age 


Our  sample  of  150  files  were  comprised  of  90  males  and  60  females. 
The  individual  ages  ranged  from  17  to  70  years  with  the  majority 
being  between  21  and  40.  Table  5  identifies  the  number  of 
individuals  in  our  sample  in  six  age  groups. 


Table  5 
Age  of  Applicant  Sample  Receiving  MVR  Services 


Age  Group 

Number  in  Sample 

— , 

Percentage  of 
Sample 

17-20 

16 

11 

21-30 

41 

27 

3  MO 

47 

32 

41-50 

32 

21 

51-60 

12 

8 

61-70 

2 

1 

Total 

150 

100 

Source:     Compiled  by  the  Legislative  Audit  Division  from 
MVR  records. 
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Occupation  at  Last 
Employment 


If  provided  on  the  application,  we  gathered  the  applicant's  last 
occupation  prior  to  applying  for  VR  services.  Based  upon  input 
from  the  Montana  Department  of  Labor  and  Industry,  we  separated 
occupations  into  six  categories. 

Service  -  Janitors,  nurses,  waitresses.,  etc. 

Clerical  -  Clerks,  cashiers,  etc. 

Trade  -  Meat  cutters,  electrical  helpers,  construction  workers,  etc. 

Agriculture  -  Forest  service,  orchard  caretakers,  ranch  hands,  etc. 

Student  -  Someone  in  high  school. 

Unemploved 

Most  applicants  come  from  service  industries.  Table  6  shows  the 
applicants'  occupation  prior  to  applying  for  VR  services. 


Table  6 
Last  Held  Occupation  of  Sampled  MVR  Applicants 




Occupation 

Percentage  of  Sample 

Service 

51 

Clerical 

9 

Trade 

25 

Agriculture 

4 

Student 

1 

Unemployed 

10 

Source:     Compiled  by  the  Legislative  Audit  Division  from 
MVR  records. 
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M^or  Disability  of  Clients 


A  major  disability  is  the  physical  or  mental  condition,  impairment, 
or  disease  most  responsible  for  an  individual's  work  limitation. 
Three-fourths  of  the  applicants  sampled  listed  either  orthopedic 
impairments  or  mental  and  emotional  conditions  as  their  major 
disability.  Table  7  shows  the  major  disabilities  reported  by 
applicants. 


Table  7 
Maior  Disabilitv  of  MVR  Apolicants 

Major  Disability 

Percentage 
of  Sample 

Orthopedic  impairments  (except  amputations)  such  as  back  injuries, 
cerebral  palsy,  arthritis  &  rheumatism,  muscular  dystrophy,  multiple 
sclerosis,  Parkinson's  disease,  etc. 

41 

Mental  and  emotional  conditions  such  as  psychosis,  neurosis, 
alcohol  abuse,  mental  retardation,  autism. 

34 

Leukemia,  asthma,  diabetes,  cystic  fibrosis,  epilepsy,  congenital  heart 
disease,  tuberculosis,  emphysema,  etc. 

13 

Deaf  and/or  Blind 

4 

Traumatic  brain  injury 

4 

Absence  or  amputation  of  a  limb  or  extremity 

2 

Not  listed 

2 

Source:  Compiled  by  the  Legislative  Audit  Division  from  MVR  records. 

Average  Length  of  Time 
Receiving  Services  from 
MVR 


We  documented  the  length  of  time  a  client  received  services  from  the 
MVR  Program.  The  time  began  with  the  application  date  and  ended 
with  closure  date.  The  range  of  time  varied  from  eight  days  to 
almost  seven  years.  About  70  percent  of  the  individuals  received 


Page  19 


Chapter  II  -  Vocational  Rehabilitation  Background 


services  for  less  than  two  years.  Table  8  shows  the  length  of  time 
individuals  received  services. 


Table  8 
Length  of  Time  Receiving  MVR  Services 


Year(s)  Received  Services 

Percentage  of  Sample 

0-1 

51 

l-2yr. 

21 

2  -  3  yr. 

11 

3-4yr. 

11 

4 -Syr. 

2 

5-6yr. 

4 

Source:     Compiled  by  the  Legislative  Audit  Division  from 
MVR  records. 
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Introduction 


Policies  Need  to  be 
Clarifled  or  Written 


The  goal  of  the  Montana  Vocational  Rehabilitation  (MVR)  Program 
is  to  promote  work  and  independence  for  Montanans  with 
disabilities.  To  accomplish  this  mission,  MVR  provides  services  to 
people  of  an  employable  age  with  physical  or  mental  disabilities  to 
help  them  secure  and  maintain  employment.  Services  range  from 
guidance  and  counseling  supplied  by  MVR  staff  to  a  college  or 
university  education. 

Overall,  we  believe  the  MVR  Program  meets  its  mission  of 
promoting  work  and  independence  for  Montanans  with  disabilities. 
We  reviewed  1 18  eligible  client  files.  Thirty-two  (27  percent)  were 
closed  after  the  clients  were  successfully  employed  for  60  days  and 
another  eight  individuals  received  post  employment  services  from 
MVR  and  were  employed  for  the  60  day  requirement.  We  reviewed 
employment  status  of  24  of  the  32  successfully  closed  clients  (the 
other  eight  clients  were  homemakers,  self-employed,  or  left  the 
state).  We  found  20  of  the  24  clients  (83  percent)  continued  to  be 
employed  as  of  the  first  quarter  of  1996.  In  addition,  five  of  the 
eight  clients  on  post  employment  continued  to  be  employed  as  of  the 
firstquarter  of  1996. 

During  the  audit  we  reviewed  the  processes  counselors  follow  for 
determining  services  needed,  providing  services,  and  measuring 
success.  We  believe  some  of  these  processes  could  be  more  efficient 
and  effeaive.  This  chapter  addresses  the  underlying  causes  for  some 
of  the  concerns  identified  during  audit  work.  Chapters  IV,  V,  and 
VI  discuss  our  findings  in  greater  detail. 

Prior  to  the  1992  Amendments  to  the  Rehabilitation  Act  of  1973, 
MVR  had  a  policy  manual  which  was  overly  detailed.  As  a  result  of 
the  1992  amendments,  the  MVR  staff  rewrote  the  policy  manual  widi 
emphasis  on  empowering  coimselors  to  allow  them  to  make  decisions 
on  their  own.  We  believe  management  made  the  new  policy  manual 
too  general. 

Clarifying  some  existing  policies  and  developing  additional  policies 
and  procedures  related  to  the  Individual  Written  Rehabilitation 
Program  (IWRP)  and  purchasing  would  increase  the  program's 
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efficiency  and  effectiveness.  The  following  areas  were  identified  as 
needing  policies  and/or  clarification: 


Timely  IWRP  development  (write  a  policy). 

Vocational  goals  and  objectives  consistent  with  individual's 

abilities  (clarify  policy). 

Aimual  and/or  periodic  IWRP  review  (clarify  policy). 

Amendments  to  IWRPs  (clarify  policy). 

Review  of  cost  information  for  providers  (write  a  policy). 


Counselors  are  required  to  make  criterial  decisions  which  affect  a 
client's  future.  The  end  result  of  these  decisions  reflect  the  NfVR 
Program's  efficiency  and  effectiveness.  Defined  policies  provide 
counselors  the  guidance  needed  to  make  decisions.  Limited  policies 
in  the  above  areas  contributed  to  the  inconsistencies  and  problems 
we  discuss  in  Chapters  FV  and  V. 


Additional  Controls 
Need  to  be  Implemented 


Once  policies  are  established,  additional  controls  will  ensure 
decisions  are  consistent  and  measure  counselors'  and  program 
performance.  These  controls  include  training  and  supervisory 
review  of  closed  files. 


Further  Training  Needed 


DPHHS  has  an  established  training  program,  and  counselors 
received  training  when  the  new  policy  manual  was  written,  but  we 
identified  areas  where  additional  training  would  be  beneficial. 
Counselors  should  be  trained  on  the  policies  we  identified  above, 
and  in  the  areas  of  eligibility  criteria  and  development  of  measurable 
requirements  in  the  IWRPs.  Training  would  help  ensure  counselors' 
decisions  are  made  within  the  boundaries  of  the  new  policies  and  are 
consistent. 


Review  Closed  Cases 


Once  policies  are  in  place  and  training  is  complete,  central  and 
regional  office  management  should  continue  to  periodically  review 
counselors'  work  to  ensure  policies  are  followed.  Counselors 
average  100  to  120  open  files  at  any  point  in  time.  Regional 
Administrators  annually  review  a  random  sample  of  15  client  files 
assigned  to  each  counselor.  Results  of  the  file  review  are  used  in  the 
counselor's  yearly  performance  evaluation.  Central  office 
management  annually  review  an  additional  random  sample  of  IS 
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files  in  each  office.  The  office's  15  files  are  used  to  evaluate  the 
overall  performance  of  the  office  and  ultimately  the  program. 

The  majority  of  files  reviewed  at  both  levels  are  open  (active)  cases. 
Very  few  closed  files  are  evaluated.  By  only  looking  at  open  cases, 
MVR  management  caimot  obtain  a  complete  picture  of  a  counselor's 
or  office's  performance.  A  review  of  more  closed  files  would  allow 
management  to  determine  how  a  counselor  is  dealing  with  every 
aspect  of  the  program,  from  application  to  closure  and  post 
employment.  If  problems  are  found  during  the  review,  management 
can  then  develop  the  necessary  supervisory  guidance  and/or  training 
to  alleviate  the  identified  problems. 


Orsanizational  Controls  ^^  above  issues  deal  with  specific  processes  of  the  MVR  Program. 

Need  to  be  Reviewed  Implementing  the  recommendations  in  Chapters  FV  and  V,  which 

address  the  above  issues,  would  help  increase  program  efficiency. 
To  increase  effectiveness,  MVR  management  should  review 
organizational  controls  which  include  goals  and  objectives, 
management  information,  and  performance  measures. 

The  MVR  Program  goals  and  management  information  are  driven 
primarily  by  federal  funding  requirements  and  number  of  cases 
closed  when  a  person  is  considered  successfiilly  rehabilitated 
(working  for  60  days).  Current  goals  are  numbers  derived  from  past 
history  and  include  generalized  "value"  statements.  MVR 
management  information  is  based  on  federal  reporting  requirements. 
Neither  of  these  approaches  help  MVR  management  to  measure 
overall  program  effectiveness.  Program  goals  should  relate  to  the 
mission  and  purpose  of  the  program.  Objectives  should  relate  to  the 
goals  and  should  describe  specific  and  measurable  targets,  results,  or 
outcomes.  Management  information  should  provide  a  mechanism  to 
gather  pertinent  information  for  use  in  determining  whether  the 
program's  goals  and  objectives  are  met. 

Management  information  also  allows  program  staff  to  develop 
performance  measures  which  help  determine  program  success. 
Overall,  performance  measures  can  help  evaluate,  fine  tune,  and 
improve  program  operations  and  assist  management  in  plaiming  for 
the  future.  Because  MVR  is  primarily  driven  by  federal  funding. 
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management  has  not  identified  the  development  of  performance 
measures  as  a  priority  for  the  program's  operations.  Overall,  we 
believe  the  MVR  Program  could  be  more  effective  by  implementing 
more  specific  organizational  controls.  Our  specific 
recommendations  are  discussed  in  Chapter  VI. 


Summary  Audit  work  showed  the  MVR  Program  is  meeting  its  mission  of 

promoting  work  and  independence  for  Montanans  with  disabilities. 
The  department  began  implementing  the  majority  of  the 
recommendations  in  the  following  three  chapters  during  the  audit. 
We  believe  the  program's  efficiency  and  effectiveness  will  increase 
when  the  recommendations  are  fully  implemented.  By  increasing  the 
efficiency  and  effectiveness,  the  program  could  increase  the  number 
of  clients  categorized  as  "closed  rehabilitated,  employed  successfully 
for  at  least  60  days." 
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Introduction 


This  chapter  summarizes  the  steps  individuals  follow  when  they 
apply  for  and  receive  vocational  rehabilitation  services.  The 
following  sections  discuss  the  application  and  eligibUity  processes, 
financial  summary  completion,  and  income  verification.  Also 
discussed  is  the  development,  revision,  and  review  of  the 
Individualized  Written  Rehabilitation  Program  (TWRP).  Finally,  we 
discuss  the  process  staff  follow  for  closing  case  files.  Throughout 
the  chapter  we  explain  our  audit  findings  and  recommendations  for 
improvements  to  the  NfVR  Program. 


Applications  for  Services 
Completed 


The  fundamental  purpose  of  the  MVR  Program  is  to  provide 
vocational  rehabilitation  services  to  disabled  individuals  so  they  may 
prepare  for,  achieve,  and  retain  an  employment  outcome.  An 
individual  can  be  referred  to  MVR  from  a  variety  of  sources 
including: 


Family 

Friends 

Federal  agencies  such  as  Social  Security  and  Veterans 

Administration 

Public  and  private  mental  health  centers 

Other  state  agencies  such  as  Job  Service  and  Workers' 

Compensation 

Schools 

Public  and  private  rehabUitation  centers 

Doctors 

Lawyers 


Individuals  must  complete  an  application  which  requests  information 
including  name,  address,  referral  source,  other  agencies  the 
individual  may  be  working  with,  the  individual's  disability(ies),  type 
of  medical/psychological  treatment  the  individual  has  received,  work 
history,  and  educational  backgroimd.  At  the  time  an  application  is 
completed,  the  individual  also  completes  a  financial  summary  which 
identifies  family  size,  as  well  as  income  and  expense  sources  and 
amounts. 
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Eligibility  Determined 


Federal  rules  state  no  applicant  can  be  excluded  or  found  ineligible 
strictly  because  of:   1)  a  certain  type  of  disability,  2)  needs  or 
anticipated  cost  of  services  required,  or  3)  the  income  level  of  an 
applicant  or  an  applicant's  family.  MVR  staff  evaluate  and 
determine  eligibility  using  the  information  on  the  application, 
medical  records,  and  federal  guidelines.  Federal  guidelines  include 
the  following  basic  requirements: 


•-       A  qualified  person  concludes  the  applicant  has  a  physical  or 
mental  impairment. 

►■        The  applicant's  physical  or  mental  impairment  constitutes  or 
results  in  a  substantial  impairment  to  employment. 

*■       The  applicant  can  benefit  in  terms  of  an  employment  outcome 
from  MVR  services. 

►■       The  applicant  requires  NfVR  services  to  prepare  for,  enter, 
engage  in,  or  retain  gainful  employment  consistent  with  the 
applicant's  strengths,  resources,  priorities,  concerns,  abilities, 
capabilities,  and  informed  choice. 

To  determine  the  applicant's  physical  or  mental  impairment, 
counselors  review  existing  medical  and/or  psychological  reports.  If 
there  are  no  existing  reports  for  review,  MVR  has  necessary  tests 
completed  by  the  client's  choice  of  provider  to  determine  the 
individual's  disability  and/or  capabilities. 


Ensure  Eligibility  is 
Determined  and 
Documented  Consistent 
with  Federal  Eligibility 
Requirements 


It  is  the  responsibility  of  the  vocational  rehabilitation  counselors  to 
determine  eligibility  for  vocational  rehabilitation  services  and 
adequately  document  that  decision.  Results  of  our  audit  work  found 
cases  where  MVR  counselors  did  not  determine  eligibility 
consistently  with  federal  eligibility  requirements  or  did  not 
document  their  decisions.  For  example: 


In  August  199S,  an  individual  missing  a  limb  requested  help 
from  MVR  to  fund  a  two-year  training  program.  The 
counselor  determined  this  person  was  not  eligible  for  services 
because  "The  individual's  limitations  did  not  cause  a 
substantial  impediment  to  employment."  The  individual  was 
seasonally  employed  doing  ranch  work,  so  the  counselor 
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concluded  the  person's  disability  did  not  prevent  him  from 
working. 

►       An  individual  with  a  back  injury  applied  for  services  from 
MVR.  This  individual  was  employed,  yet  wanted  to  become 
more  "self-sufficient"  and  requested  MVR  help  to  start  his  own 
business.  The  individual  was  determined  eligible  for  services. 

Audit  work  found  instances  of  inconsistent  application  of  federal 
eligibility  requirements  involving  services  to  transitioning  students. 
Federal  rule  defines  a  transitioning  student  as  a  student  eligible  to 
receive  vocational  rehabilitation  services  and  is  receiving  transition 
services. 

"Transition  services  are  a  coordinated  set  of  activities  for 
a  student  designed  within  an  outcome  oriented  process 
which  promotes  movement  from  school  to  post  school 
activities,  including  post  secondary  education,  vocational 
training,  integrated  employment  (including  supported 
employment),  continuing  and  adult  education,  adult 
services,  independent  living,  or  community 
participation." 

The  primary  role  of  transition  services  is  one  of  planning  for  the 
student's  years  after  leaving  high  school. 

File  review  identified  an  applicant  who  applied  for  services  in 
October  1994  and  was  denied  transition  services  from  MVR.  The 
counselor  decided  the  applicant  was  ineligible  for  transition  services 
because  he/she  had  seven  months  of  high  school  remaining.  At  the 
same  regional  office,  as  well  as  at  other  regional  offices  visited, 
transition  services  were  provided  to  students  currently  in  high 
school. 


Controls  Needed  to  Ensure 
Consistent  Application  of 
Requirements  and  Proper 
Documentation 


Based  upon  interviews  with  MVR  management,  authority  for  most 
decisions,  including  eligibility,  is  given  directly  to  the  individual 
counselors.  Audit  work  identified  limited  controls  ensuring 
decisions  are  consistent  with  federal  requirements  or  are  properly 
documented.  Without  adequate  supervisory  review  and  training, 
inconsistent  application  of  federal  requirements  as  identified  during 
our  file  review  could  result.  Supervisory  review  of  a  sample  of 
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closed  ineligible  files  and  training  for  counselors  will  help  ensure 
consistent  determination  and  proper  documentation.  Supervisory 
reviews  help  identify  inconsistencies  and  determine  strengths  and 
weaknesses.  Training  provides  staff  with  the  framework  necessary 
to  consistently  determine  eligibility. 


Program  Action:  Policy 
Drafted  and  Training 
Planned 


MVR  established  a  draft  policy  which  defines  consultation  and 
inventorying  levels  of  counselors,  peers  and  supervisors  in  case 
closure  rational  and  eligibility  determination.  A  draft  policy  has 
been  developed  outlining  numbers  of  closed  files  to  be  aimually 
reviewed.  Training  to  address  this  recommendation  will  be  included 
in  MVR's  aimual  eligibility  threshold  training. 

Recommendation  #1 

We  recommend  Montana  Vocational  Rehabilitation  Program 
management: 

A.  Complete  supervisory  reviews  of  a  sample  of  closed  files, 
including  those  closed  because  the  applicant  was  det^tnined 
ineligible. 

B.  Train  staff  cm  eligibility  criteria  to  ensure  consistent  appli- 
ll    cation  of  fiederal  requirements  and  proper  documentation. 


60-Day  Eligibility 
Requirement 


Federal  guidelines  require  MVR  personnel  to  determine  if  an 
individual  is  eligible  or  ineligible  within  60  days  of  application  for 
services.  The  60-day  regulation  is  designed  to  avoid  unnecessary 
delays  in  clients  receiving  services.  Regulations  provide  for 
exceptions  to  the  60-day  requirement  in  the  event  there  are 
unforeseen  circumstances  beyond  the  agency's  control.  If  there  are 
unforeseen  circumstances,  it  is  the  counselor's  responsibility  to 
document  the  need  for  an  extension  of  time,  then  place  the 
individual  in  an  extended  evaluation  status  w^ich  may  not  exceed  18 
months.  Federal  rule  further  stresses  the  importance  for  states  to 
make  "good  faith  efforts"  to  obtain  necessary  information  to 
determine  eligibility  as  quickly  as  possible.  The  60-day  requirement 
became  effeaive  in  November  1993. 
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Of  the  150  files  reviewed,  91  applications  were  submitted  on  or  after 
November  1993.  We  tested  these  applications  to  evaluate  if  MVR 
counselors  determined  eligibility  within  the  federal  60-day 
requirement.  If  a  file  contained  documentation  of  an  unforeseen 
circumstance,  such  as  a  delay  in  obtaining  necessary  reports,  the  file 
was  excluded  from  our  review.  We  excluded  two  files  in  our 
sample.  Results  of  our  review  showed  26  of  89  (30  percent)  client 
files  sampled  did  not  have  eligibility  determined  within  the  60-day 
federal  requirement.  Table  9  provides  results  of  our  testing. 


Table  9 
Time  Frame  for  Determining  MVR  Eligibility 


Time  Frame 

Number  of 
Applicants 

Percent  of 
Sample 

60  days  or  less 

63 

71 

Greater  than  60  days 

26 

29 

Total 

89 

100 

Source:  Compiled  by  the  Legislative  Audit  Division 

from  MVR  records. 


The  recent  federal  review  of  MVR  also  identified  instances  of  non- 
compliance with  the  60-day  requirement. 

As  mentioned  previously,  we  identified  discrepancies  with  the  MVR 
information  system  in  place  during  the  time  the  above  individuals 
submitted  their  applications.  Some  of  the  noncompliance  identified 
above  may  have  resulted  from  the  system  not  providing  management 
with  accurate  monitoring  information.  Reports  supervisors  used  to 
determine  the  length  of  time  from  application  to  eligibility 
determination  were  not  based  on  application  date  but  rather  on  the 
date  the  information  was  input  into  the  system. 

MVR  should  ensure  the  new  computer  system  provides  staff  with  the 
information  necessary  to  ascertain  if  clients  are  determined  eligible 
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or  ineligible  within  the  appropriate  time  lines.  This  information  will 
ultimately  result  in  clients  receiving  applicable  services  in  a  timely 
manner. 


Program  Action:  Cases 
Exceeding  60  Days  to  be 
Reviewed 


MVR  drafted  a  policy  addressing  timeliness  of  eligibility 
determinations.  This  policy  will  require  supervisory  review  of  all 
cases  which  exceed  the  60-day  eligibility  time  frame.  MVR  also 
indicated  training  is  being  planned  to  assure  compliance. 


Recommendation  #2 

We  recommend  Montana  Vocational  Rehabilitation  Program 
management  provide  staff  with  information  to  help  ensure 
compliance  with  the  federal  60-day  time  frame  for  eligibility 
determination. 


Individualized  Written 
Rehabilitation  Program 
Process 


Once  an  applicant  is  determined  eligible  for  vocational  rehabilitation 
services,  an  Individualized  Written  Rehabilitation  Program  (IWRP) 
is  developed.  The  IWRP  is  developed  by  vocational  rehabilitation 
counselors  in  collaboration  with  the  client.  The  1992  Amendments 
to  the  Rehabilitation  Act  emphasize  the  following  requirements 
specific  to  an  IWRP: 


»■      The  IWRP  must  be  developed  with  reasonable  long-term 
vocational  goals,  and  contain  specific  and  measurable 
rehabilitation  objectives. 

•■      The  vocational  goals  and  objectives  outlined  in  the  IWRP  must 
lead  to  an  employment  outcome  consistent  with  an  individual's 
strengths,  priorities,  concerns,  abilities,  capabilities,  and  career 
interests. 

••      The  IWRP  must  provide  individuals  the  necessary  information 
to  make  an  informed  choice  concerning  specific  goods  and 
services,  including  the  providers  of  goods  and  services. 

MVR  policy  further  requires  an  IWRP  to: 
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►      Outline  rehabilitation  objeaives  relating  to  attainment  of  the 
employment  goal,  including  objective  criteria  and  an  assessment 
procedure  to  determine  whether  goals  and  objectives  are 
achieved. 

►■      Contain  a  list  of  specific  vocational  rehabilitation  goods  and 
services  to  be  provided,  and  projected  dates  for  the  initiation 
and  duration  of  each  good  and  service. 

••  Be  periodically  reviewed  to  determine  whether  goals  and 
objectives  are  met  and  if  services  accomplish  the  outlined 
objectives. 

►■      Be  reviewed  annually.  At  which  time,  if  necessary,  an 
amendment  or  revision  to  the  program  may  result. 

•■      Be  amended  or  revised  anytime  there  is  a  significant  change  in 
the  individual's  financial  status  or  life  circumstance. 

Both  the  counselor  and  the  individual  receiving  services  from  MVR 
use  the  IWRP  because  it  outlines  the  goals  and  objectives  necessary 
to  lead  to  an  employment  outcome. 


Conclusion:  IWRPs 
Designed  with  Employment 
Outcomes 


According  to  federal  and  state  guidelines,  each  IWRP  must  be 
designed  to  achieve  an  employment  objective  consistent  with  the 
strengths,  resources,  priorities,  concerns,  abilities,  and  capabilities 
of  the  individual.  We  found  all  applicable  IWRPs  reviewed  were 
designed  with  employment  as  the  outcome.  Examples  of 
employment  outcomes  included  occupations  such  as  a  clothes  hanger 
at  a  sheltered  workshop,  a  fast  food  worker,  an  administrative 
support  person  for  a  state  agency,  a  licensed  practical  nurse,  and  a 
teacher. 


Timely  Development  of 
IWRPs  Important 


Timely  development  of  IWRPs  helps  ensure  clients  receive  necessary 
services  promptly  and  help  prepare  the  individual  for  achievement  of 
a  vocational  outcome.  According  to  federal  rule,  the  program's 
intent  is  that  clients  receive  services  as  "quickly  as  possible"  and  it  is 
important  for  vocational  rehabUitation  programs  to  "guard  against 
delays  in  the  development  of  the  IWRP  once  an  individual  is 
determined  eligible  for  services." 


File  review  identified  delays  associated  with  development  of  IWRPs 
once  eligibUity  was  determined.  Of  the  150  case  files  reviewed,  85 
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had  initial  IWRPs  which  we  evaluated  to  determine  if  they  were 
developed  in  a  timely  manner.  If  the  counselor's  notes  documented 
reasons  justifying  a  delay  in  development  of  an  IWRP,  such  as 
waiting  for  results  of  testing,  we  excluded  those  IWRPs  from  our 
evaluation.  We  calculated  the  time  to  develop  an  IWRP  by  using  the 
date  eligibility  was  determined  and  the  date  of  the  initial  IWRP.  We 
were  able  to  evaluate  83  of  the  85  IWRPs  for  timely  development. 
Two  IWRPs  were  excluded  from  the  evaluation  because  we  were 
unable  to  verify  the  dates  the  IWRPs  were  initially  written.  Our 
review  of  the  83  IWRPs  found  it  took  counselors  two  months  or  less 
to  develop  45  of  the  83  IWRPs  (54  percent).  It  took  counselors  two 
months  to  more  than  one  year  to  develop  38  of  the  83  IWRPs 
(46  percent).  Table  10  shows  the  length  of  time  it  took  to  develop 
IWRPs  in  our  sample. 


Table  10 
Length  of  Time  to  Develop  TWRPs 

Time  Frame 

Number  of 
Files 

Percent  of 
Sample 

2  months  or  Less 

45 

54 

2.1  months -6  mos. 

29 

35 

6.1  months- 1  year 

7 

8 

Greater  than  1  year 

2 

3 

TOTAL 

83 

100 

Source:      Compiled  by  Legislative  Audit  Division  from 
MVR  records. 

Federal  rules  indicate  time  lines  for  development  of  IWRPs  should 
operate  as  guidelines  and  should  be  consistent  with  legislative  intent. 
MVR  management  has  not  formally  established  any  time  lines  nor 
any  policy  to  address  timely  development  of  IWRPs.  Discussions 
with  management  indicated  IWRPs  should  be  developed  in  two 
months  but  this  policy  has  not  been  formally  developed  or 
communicated  to  staff. 
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Although  there  is  no  policy  addressing  timely  development, 
supervisors  review  counselor  files  to  determine  if  IWRPs  are 
developed  in  a  timely  manner.  Supervisors  evaluate  timeliness  of 
IWRP  development  using  the  definition  of  timely  manner  in 
appendix  D  of  the  MVR  policy  manual.  Appendix  D  defines  timely 
manner  as  ".  .  .the  point  of  reference  in  that  the  case  narrative  must 
demonstrate  the  individual  was  either  actively  engaged  in  case 
activity,  waiting  for  reasonable  lengths  of  time  for  scheduled 
appointments,  or  jointly  focused  with  the  counselor  on  an  anticipated 
training  schedule  (i.e.,  preparing  for  a  spring  semester)."  "Timely 
manner"  and  "reasonable  lengths  of  time"  are  not  clearly  defined, 
leaving  the  definitions  open  to  individual  interpretation  by  all  staff. 
Without  controls  in  place  to  ensure  IWRPs  are  developed  and 
implemented  in  a  timely  manner,  there  is  limited  assurance  services 
are  provided  as  "quickly  as  possible"  to  clients. 

MVR  management  should  develop  time  lines  to  address  prompt 
development  of  IWRPs.  Once  time  lines  are  established,  policy 
should  be  written.  Policy  can  help  ensure  clients  receive  services 
promptly  and  allows  supervisors  to  evaluate  timeliness  against  an 
established  standard. 

Program  Action:  Time  MVR  wrote  a  policy  which  requires  the  IWRP  to  be  written  within 

Frames  Established  six  months  of  application.  If  the  client  and  supervisor  agree  an 

extension  of  time  is  warranted,  the  rational  shall  be  documented. 

All  files  without  an  IWRP  six  months  after  application  will  be 

reviewed  by  regional  management. 


Rfecommendgtion  #3 

We  recomm&ad  Montana  Vocational  Rehabilitatiim  Program 
management  establish  and  implement  time  lines  for  development 
of  IWRPS. 
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Reasonable  Objectives  and 
Measurable  Requirements 
Are  Important 


Federal  guidelines  require  the  IWRP  contain  specific  rehabilitation 
objectives  to  attain  the  individual's  long-term  vocational  goal.  In 
addition,  MVR  policy  requires  the  IWRP  to  have  reasonable 
objectives  and  measurable  requirements.  Both  the  counselor  and  the 
client  develop  these  objectives  and  requirements  to  ensure  everyone 
understands  their  respective  responsibilities. 


Conclusion:  Objectives 
Appeared  Reasonable 


An  objective  must  answer  the  question  what  does  this  person  want  to 
accomplish?  According  to  MVR  policy  the  IWRP  must  be  designed 
to  achieve  the  employment  objective  of  the  individual.  Evaluation 
of  the  objectives  in  the  IWRPs  sampled  found  all  objectives  appeared 
reasonable.  Examples  of  objectives  in  files  we  reviewed  included: 


Requirements  Should  be 
Speciflc  and  Measurable 


►■      To  do  work  on  own. 

•■      To  become  a  mill  worker. 

►•      To  obtain  a  degree  in  counseling. 

We  then  evaluated  the  requirements  or  steps  developed  by  the 
counselors  and  clients  to  determine  if  the  requirements  were  specific 
and  measurable.  Measurable  requirements  provide  the  direction 
necessary  to  achieve  an  objective  outlined  in  the  IWRP.  Measurable 
requirements  also  outline  the  client's  and  counselor's  responsibilities 
for  attaining  the  objective.  For  example,  if  an  established  objective 
is  to  obtain  a  college  education,  measurable  requirements  could  be  to 
maintain  adequate  attendance,  maintain  a  B  grade  point  average, 
provide  the  counselor  with  copies  of  grades,  and  maintain  contact 
with  the  counselor  quarterly  or  on  a  semester  basis. 


File  review  found  1 1  of  85  IWRPs  (12.9  percent)  did  not  have 
measurable  requirements  outlined  in  the  IWRP.  For  example,  an 
IWRP  developed  in  1994  identified  an  objective  to  alleviate 
depression,  in  hopes  of  obtaining  an  employment  outcome.  The 
requirements  stated  the  individual  was  to  maintain  contact  with  the 
counselor,  yet  nothing  quantitatively  outlined  how  the  counselor 
would  measure  the  outcome.  We  identified  another  IWRP  developed 
in  1994  with  the  main  objective  to  increase  self-sufficiency  and 
independence.  The  requirements  were  for  MVR  to  provide  guidance 
and  counseling,  yet  MVR  provided  funding  for  computer  software 
and  hardware  to  start  the  individual's  own  business.  The 
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requirements  were  not  specific  to  what  MVR  actually  did  and  also 
did  not  establish  a  level  or  measurement  of  achievement. 

Discussion  with  program  staff  indicated  it  is  difficult  to  write 
measurable  requirements.  Management  said  there  should  be  on- 
going training  addressing  development  of  measurable  requirements 
in  the  IWRP.  To  ensure  training  is  effective  and  policy  is  followed, 
management  should  review  a  sample  of  IWRPs. 


Program  Action:  IWRP 
and  Case  Review  Form 
Revised 


MVR  revised  both  the  IWRP  and  the  case  review  instrument  to  focus 
the  counselor  more  accurately  in  terms  of  developing  "measurable" 
performance  criteria  for  evaluating  the  IWRP  and  goals. 


Recommendation  #4 

We  recommend  Montana  Vocational  Rehabilitation  Program 
management  ensure  counselors: 

A.  Receive  appropriate  training  for  development  of  measurable 
requirements. 

B.  Develop  measurable  requirements  by  reviewing  IWRPs. 


Ensure  Vocational  Goals 
and  Objectives  are 
Consistent  with  Abilities 


Federal  rule  requires  an  individual  to  make  an  informed  choice 
regarding  selection  of  his/her  vocational  goals  and  objectives.  In 
addition,  federal  regulations  require  the  goals  and  objectives  be 
consistent  with  the  individual's  strengths,  priorities,  concerns, 
abilities,  capabilities,  and  career  interest.  State  policy  requires  each 
counselor  "...  to  respect  the  client  as  an  individual  who  has  the 
right  and  responsibility  to  participate  in  all  decisions  regarding  his 
or  her  vocational  future. "  MVR  staff  and  management  interpret  this 
to  mean  the  major  emphasis  of  the  program  is  the  individual's 
choice,  i.e.,  the  choice  of  an  individual  is  primary.  Counselors 
indicated  if  an  individual  is  eligible  for  services  with  MVR,  it  is 
usually  MVR's  responsibility  to  provide  the  services  the  individual 
requests. 
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File  review  verified  client  choice  has  taken  precedence  over  making 
informed  choice.  In  12  of  76  files  (15.8  percent)  we  found 
individuals  received  training  and/or  services  which  were  not 
consistent  with  their  abilities  as  determined  by  evaluations  and 
vocational  testing.  For  example,  MVR  paid  for  four  years  of  higher 
education,  beginning  in  1992,  for  an  individual  to  become  a 
counselor.  The  initial  psychological  evaluation  identified  concerns 
with  this  type  of  occupation.  Throughout  the  years  this  client 
received  MVR  services,  staff  identified  concerns  with  the  occupation 
choice.  A  short  time  after  the  individual  obtained  employment  as  a 
counselor  it  was  discovered  by  the  employer  the  individual  was 
incapable  of  fulfilling  responsibilities  of  the  position.  The  individual 
was  terminated  from  employment. 

Another  individual  applied  for  MVR  services  in  early  1993  and  was 
determined  eligible.  Vocational  testing  showed  the  individual  did 
not  have  the  functional  capacity  to  fulfill  responsibilities  of  the 
chosen  profession.  MVR  staff  also  documented  concerns  with  the 
ability  of  the  client  to  obtain  employment  upon  completion  of  the 
training.  According  to  the  counselor,  since  the  client  had  already 
paid  for  the  first  year  of  training  on  his  own,  MVR  paid  for  his 
second  year  of  training.  Upon  completion  of  the  training  program, 
it  was  evident  to  both  the  individual  and  MVR  the  client  was  unable 
to  enter  the  career  plaimed  for  by  the  client,  and  endorsed  and 
purchased  by  MVR. 


Counselor's  Responsibility 
to  Help  Individual  Make 
Informed  Choices 


The  position  description  for  a  rehabilitation  counselor  requires  a 
counselor  to  be  knowledgeable  of  the  principles  and  techniques  of 
vocational  evaluation,  guidance,  counseling,  and  job  placement.  It 
is  the  counselor's  responsibility  to  evaluate  an  individual's  strengths 
and  abUities  to  help  the  individual  make  informed  choices  as  to  his 
or  her  vocational  goals,  objectives  and  finally  an  employment 
outcome.  For  example,  the  counselor  should  provide  the  client 
career  choices  which  coincide  with  the  client's  strengths.  A  client 
may  benefit  from  a  two-year  training  course  rather  than  a  four-year 
training  program,  or  on-the-job  training  may  be  more  realistic  for 
the  client.  If  individuals  are  aware  of  their  shortcomings  and  their 
strengths,  they  can  make  informed  choices  as  to  their  future  career 
plans. 
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We  identified  cases  where  counselors  advised  individuals  to 
reconsider  an  employment  path  or  training  area  the  counselor 
considered  inappropriate.  These  cases  were  closed  successfully  in 
that  the  individuals  accepted  the  counselors'  recommendations  and 
were  successfully  placed.  For  example,  in  early  1994,  one 
individual  wanted  to  obtain  a  four-year  degree.  After  testing,  it  was 
discovered  the  individual  had  some  significant  scholastic 
weaknesses.  With  the  help  of  the  counselor,  the  individual 
reconsidered  the  career  goal  and  was  successfully  placed  in  a  non- 
degree  position. 

Current  MVR  policy  does  not  emphasize  the  importance  of  a  client's 
strengths,  abilities  and  capabilities.  MVR  management  should 
clarify  current  policy  to  reflect  the  importance  of  all  the  attributes 
when  planning  an  individual's  vocational  future. 


Program  Action:  Policy 
Rewritten 


MVR  is  publishing  a  policy  on  this  topic  to  provide  guidance  to 
counselors.  The  draft  policy  states  individuals  will  be  provided 
information  and  orientation  at  application  and  again  at  the  time  of 
the  development  of  the  IWRP  regarding  the  availability  and  scope  of 
informed  choice. 


Recommendation  #5 

We  recommend  Montana  Vocational  Rehabilitation  Program 
management  clarify  policy  to  help  ensure  counselors  consider  an 
Individual's  strengths,  abllitl^,  and  capabilities  when  planning 
his/her  vocational  future. 


Periodic  and/or  Annual 
Review  of  IWRPs  is 
Important 


Federal  rule  requires  state  units  to  review  IWRPs  with  clients  as 
often  as  necessary,  but  at  least  once  each  year.  This  review  is  to 
assess  an  individual's  progress  in  meeting  objectives  identified  in  the 
IWRP.  Federal  rule  also  requires  state  units  to  establish  a  procedure 
and  schedule  for  periodic  review  and  evaluation  of  FWRPs.  MVR 
supervisors  evaluate  a  sample  of  each  counselor's  case  files,  and 
during  this  evaluation  determine  if  the  counselor  has  completed  an 
aimual  review  of  the  IWRP. 
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MVR  policy  also  requires  counselors  to  document  periodic  reviews 
of  IWRPs.  Policy  states  "As  the  IWRP  commits  to  a  schedule  for 
determining  whether  goals  and  objectives  are  being  met,  case  entries 
which  address  the  assessment  schedule  shall  be  entitled  'Periodic 
Review'  and  dated." 


Not  All  Annual  Reviews 
Documented 


Fifty-one  IWRPs  in  our  sample  required  annual  reviews  since  the 
clients  received  services  for  more  than  one  year.  Twenty-two  of  5 1 
files  (43  percent)  did  not  contain  documentation  an  annual  review  of 
the  IWRP  was  completed.  For  example,  one  file  contained 
documentation  the  IWRP  was  reviewed  in  1993,  no  documentation 
of  review  in  1994,  and  finally  closure  in  mid-1995.  MVR  services 
were  provided  during  1994  and  1995.  Annual  review  of  the  IWRP 
was  not  completed,  even  though  the  counselor's  notes  identified  the 
individual  had  difficulty  meeting  requirements  outlined  in  the  IWRP 
during  1994. 


Periodic  Reviews  Need  to 
be  Documented 


We  also  tested  files  for  documentation  of  periodic  review  of  the 
IWRPs  by  counselors.  We  found  16  of  68  files  (24  percent)  did  not 
contain  documentation  counselors  completed  periodic  reviews  of  the 
IWRPs.  For  example,  a  client  receiving  post-employment  services 
was  required  to  contact  the  counselor  at  least  every  90  days. 
Because  90-day  follow-up  was  required  in  the  IWRP  we  expeaed  to 
see  documentation  of  the  contaa  in  the  counselor's  notes,  constitut- 
ing periodic  review.  The  file  did  not  contain  documentation  there 
was  contact  or  at  least  periodic  review  by  the  counselor. 


MVR  is  not  clear  as  to  what  constitutes  "periodic."  Staff  indicated 
even  though  there  is  policy  for  periodic  review,  there  has  been  no 
decision  as  to  what  periodic  means.  Periodic  is  not  defined  in  the 
policy  manual. 
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Annual  and/or  periodic  reviews  provide  a  counselor  with  a  track 
record  of  a  client's  progress.  Documentation  of  the  review  provides 
anyone  reading  the  file  a  clear  understanding  of  the  client's  status. 
Without  scheduled  documented  review  of  the  IWRP,  counselors  are 
unable  to  determine  if  clients  obtain  agreed  upon  goods  and  services 
and  whether  clients  are  working  toward  established  employment 
objectives.  This  review  would  also  help  identify  needed 
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amendments  to  the  rehabilitation  objectives  and  requirements  in  the 
IWRP. 


Program  Action:  Training 
Planned 


MVR  management  established  aimual  reviews  as  a  priority  of  case 
reviews.  To  clarify  periodic  reviews,  management  believes  the 
revised  IWRP  format  will  lend  itself  to  periodic  reviews.  MVR 
indicated  there  will  be  training  to  address  these  recommendations. 


Recommendation  #6 

We  recommend  Montana  Vocational  Rehabilitation  Program 
management: 

A.  Ensure  coonselors  are  completing  and  documenting  annual 
review  of  IWRPs. 

B.  Oarify  established  policies  to  define  periodic  review  of 
IWRPs. 


Amendments  of  IWRPs 
Important  When  Changes 
Occiu" 


Upon  completion  of  a  periodic  and/or  annual  review,  the  counselor 
can  revise  or  amend  the  IWRP  if  necessary.  According  to  MVR 
policy,  an  amendment  to  the  IWRP  should  result  if  a  client  is 
experiencing  changes  in  his/her  life  which  could  affect  his/her 
employment  goal,  rehabilitation  goals  and  objeaives,  and  goods 
and/or  services  outlined  in  the  IWRP.  Policy  also  requires  an  IWRP 
amendment  or  revision  if  there  is  a  significant  change  in  the 
individual's  financial  status  or  life  circumstance.  Since  policies  do 
not  accurately  define  what  is  considered  a  significant  change  in  a 
client's  financial  status  or  life  circumstance,  during  file  review  we 
looked  for  things  such  as  loss  of  a  job,  change  in  income,  change  in 
marital  status,  etc. 


Of  the  85  files  reviewed  with  IWRPs,  64  contained  documentation  of 
some  type  of  significant  change  affecting  the  client.  Of  the  64  files, 
we  found  29  (45  percent)  did  not  have  amendments  developed.  For 
example,  an  individual  was  working  in  a  probationary  job  after 
applying  for  MVR  services.  The  original  IWRP  was  developed  in 
April  1993.  The  individual  quit  the  probationary  job  one  month 
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later,  yet  no  amendment  was  developed  for  the  client  showing  the 
change.  Counselors'  notes  showed  MVR  provided  a  vocational 
evaluation  approximately  two  months  later.  The  next  month,  contact 
notes  showed  the  individual  received  adult  education  as  a  service. 
There  was  no  evidence  amendments  were  made  to  the  IWRP  to 
reflect  the  changes  in  the  individual's  life. 

As  mentioned  previously,  the  purpose  of  an  IWRP  is  to  document  a 
program  of  action  to  obtain  a  successful  employment  goal.  If 
significant  changes  in  a  client's  life  affea  outlined  employment 
goals,  objectives,  and/or  responsibilities  of  the  client  or  MVR,  the 
IWRP  should  be  amended.  The  IWRP  is  jointly  used  by  the 
counselor  and  the  client;  therefore,  if  changes  occur,  amendments  to 
the  IWRP  are  important.  Management  should  develop  controls  to 
ensure  amendments  are  made  when  necessary.  These  controls 
should  include  clear,  concise  policies  and  procedures.  Management 
should  also  make  sure  counselors  understand  when  written 
amendments  are  necessary. 


Program  Action:  Policy 
Drafted 


MVR  management  drafted  a  policy,  based  on  federal  regulations, 
clarifying  when  amendments  are  needed. 


Recommendation  #7 

We  recommend  Montana  Vocational  Rehabilitation  Program 
management: 

A.  Clarify  IWRP  amendment  policies  and  procedures. 

B.  Ensure  counselors  amend  IWRPs  whai  necessary. 


Client  Financial 
Information  Gathered 
and  Reviewed 


Federal  regulations  allow  state  vocational  rehabilitation  programs  to 
consider  the  financial  need  of  clients  when  deciding  the  extent  of  the 
client's  participation  in  paying  for  the  services.  MVR  has  chosen  to 
use  a  financial  needs  test  to  assess  the  client's  ability  to  contribute 
toward  the  cost  of  services  provided.  Section  53-7-108,  MCA, 
states  MVR  must  provide  counseling  and  guidance,  vocational 
evaluation,  and  vocational  placement  services  without  cost  to  an 
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eligible  person.  In  addition  to  these  services,  the  program  may  pay 
the  cost  of  other  vocational  rehabilitation  services,  such  as  training, 
for  an  eligible  person  to  the  extent  MVR  determines  the  person 
cannot  pay  for  the  cost  of  those  services. 

The  MVR  financial  need  policy  requires  counselors  to  determine  the 
extent  of  the  client's  financial  need.  MVR  requires  rehabilitation 
services  be  provided  at  public  cost  only  to  clients  found  to  require 
financial  assistance.  MVR's  IWRP  policy  requires  documentation  in 
the  IWRP  describing  the  extent  of  the  client's  participation  in  the 
cost  of  services. 

To  determine  the  client's  ability  to  help  pay  for  goods  and/or 
services,  each  client  is  required  to  complete  a  financial  summary 
form.  The  form  is  completed  at  the  time  of  application  and  again 
when  there  is  a  change  in  the  client's  financial  status.  Some  MVR 
staff  indicated  clients  not  receiving  purchased  goods  and/or  services 
are  not  required  to  complete  the  financial  summary  form  other  than 
indicating  "no  cost  plan"  on  the  form. 


Complete  Financial  At  the  beginning  of  the  audit,  MVR  staff  stated  the  financial 

Summaries  as  Required  summary  forms  were  completed  annually  or  more  frequently  as 

needed.  During  the  audit,  the  department  implemented  a  new  policy 
which  requires  updating  financial  sunmiary  forms  only  as  needed. 

During  our  file  review,  we  recorded  how  often  financial  summary 
forms  were  completed.  Files  closed  because  the  individual  was  not 
eligible  for  MVR  services  were  excluded  in  our  analysis  of  financial 
summary  forms,  resulting  in  1 18  files  with  259  summary  forms. 
Overall,  35  percent  of  the  financial  summaries  were  not  completed 
within  one  year.  Table  1 1  shows  the  frequency  of  financial 
summary  forms  completed  in  our  sample. 
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Table  11 
Completion  of  MVR  Financial  Summary  Forms 


Time  Period  Completed 

Number 
of  Forms 

Percent 
of  Forms 

12  or  months  or  less 

169 

65 

More  than  12  months 

90 

35 

Total 

259 

100 

Source:       Compiled  by  the  Legislative  Audit  Division  from 
MVR  records. 


As  a  result  of  not  requiring  clients  to  complete  financial  summaries 
annually  or  more  frequently  as  needed,  MVR  is  not  aware  of  client 
resources.  These  resources  could  be  used  to  help  pay  for  MVR 
purchased  goods  and  services.  We  found  instances  where  updated 
financial  summary  forms  were  not  completed  after  there  were 
changes  in  a  client's  financial  situation.  For  example,  files 
contained  documentation  of  increased  parental  support,  additional 
spouse  wages,  increased  Workers'  Compensation,  increased  Social 
Security  income,  student  financial  aid  awards,  employment  and  wage 
changes,  and  changes  in  living  expense  due  to  moving.  We  also 
found  examples  of  clients  returning  for  post-employment  without 
completing  new  or  updated  financial  summary  forms.  Overall,  we 
identified  at  least  13  percent  of  the  tiles  with  linancial  summary 
forms  were  not  updated  as  needed.  We  based  this  on  information  in 
the  coimselor  notes  or  information  contained  in  the  IWRP. 

Staff  opinions  and  attitudes  concerning  client  financial  resources 
appeared  to  contribute  to  financial  summaries  not  being  completed  as 
policy  requires.  MVR  counselors,  counselor  supervisors  and  region 
administrators  interviewed  believe  the  majority  of  clients  cannot 
afford  to  participate  in  the  cost  of  goods  and  services.  Another 
contributing  factor  may  be  limited  supervisory  review  of  closed 
cases.  By  not  reviewing  closed  cases,  MVR  has  limited  management 
information  to  ensure  financial  sunmiaries  were  completed  as 
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required  by  policy.  We  found  instances  where  financial  summary 
forms  were  completed  at  application  and  a  year  later,  but  no  update 
for  the  next  several  years  through  case  closure,  resulting  in  long 
time  spans  when  financial  information  was  not  current. 


Limited  Documentation  of  Under  the  assumption  financial  information  is  used  to  determine 

Client  Contributions  client  participation  in  purchasing  goods  and  services,  we  reviewed 

financial  summary  forms  in  our  sampled  files.  We  documented  the 
clients'  monthly  income  and  expenses  as  reported  on  these  forms. 
Available  income  was  calculated  by  subtracting  reported  client 
expenses  ft-om  reported  client  income.  The  sampled  clients'  reported 
monthly  available  income  ranged  from  none  up  to  $959.  Most 
clients  reported  no  money  available  to  contribute  toward  the  cost  of 
goods  and  services  purchased  by  MVR. 

In  addition  to  our  review  of  the  financial  sunamary  forms,  we 
reviewed  other  file  documentation  to  determine  the  extent  of  client 
contributions.  We  found  limited  documentation  of  clients'  ability  or 
inability  to  financially  contribute  to  services  or  goods  purchased  by 
MVR. 

Without  documenting  clients'  ability  to  make  contributions,  MVR 
cannot  determine  the  extent  of  an  individual's  participation  in  the 
cost  of  goods  and  services.  MVR  also  cannot  determine  the  extent 
of  client  contribution  for  any  group  of  clients  (such  as  region 
comparison)  or  over  a  period  of  time  (such  as  fiscal  year 
comparison).  Without  compiled  information  on  client  contributions, 
management  cannot  assess  the  success  of  or  the  need  for  client  cost 
participation. 

Since  MVR  has  chosen  to  require  financial  assessment  and  client  cost 
contribution,  MVR  should  ensure  consistent  financial  assessment  and 
payment  for  goods  and  services  for  clients  found  to  require  financial 
assistance.  MVR  needs  to  communicate  the  importance  of  financial 
assessment  and  client  contribution  documentation  requirements  to 
staff.  This  can  be  accomplished  through  written  policy,  training  on 
the  policy,  and  reviewing  closed  files.  The  latter  will  ensure  policies 
are  applied  consistently  and  identify  any  further  training/clarification 
needed. 
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Program  Action:  Financial 
Policy  Strengthened 


MVR  revised  and  strengthened  its  financial  policy.  They  also 
developed  policies  addressing  annual  reviews  of  financial  status  and 
the  need  to  amend  the  financial  report  and  cost  sharing  arrangements 
as  appropriate.  MVR  said  they  will  review  closed  files  to  monitor 
compliance  with  policies. 


Recommendation  ^S 

We  recommend  Montana  Vocational  Rehabilitation  Program 
management: 

A.  Emphasize  the  need  for  financial  summaries  and 
documentation  of  client  contributions  through  policy  and 
training. 

B.  Review  closed  files  to  monitor  compliance  with  the  policies. 


Client  Income  Not  Verified 


MVR  does  not  verify  income  reported  by  clients  on  the  financial 
summary  form.  MVR  policy  indicates  the  client  is  the  primary 
source  of  financial  information,  but  counselors  can  determine 
financial  resources  from  any  reliable  source.  The  counselor  may  not 
expend  program  funds  for  services  for  any  client  who  fails  to 
provide  necessary  financial  information.  Staff  believe  MVR  clients 
accurately  report  financial  information. 


During  our  audit  work  we  verified  wages  reported  by  MVR  clients 
in  our  file  sample.  We  excluded  closed  files  of  applicants  not 
eligible  for  MVR  services.  We  verified  wages  from  the 
Unemployment  Insurance  (UI)  computer  system  for  80  clients  closed 
between  October  1994  and  June  1995,  since  the  system  had  wages 
for  that  time  period.  We  found  17  of  80  clients  (21  percent)  had 
income  greater  than  the  amount  reported  on  the  financial  summary 
forms.  For  the  17  clients,  we  identified  an  additional  $7,740  in 
monthly  wages  not  reported  on  financial  summaries  or  in  a  financial 
summary  update. 

Since  MVR  chose  to  require  financial  needs  assessment  of  clients  to 
determine  a  client's  ability  to  participate  in  the  cost  of  goods  and 
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services,  MVR  should  establish  controls  to  help  ensure  clients  report 
accurate  and  complete  financial  information.    To  be  cost-effective 
and  display  the  existence  of  controls,  MVR  could  verify  income  for 
a  sample  of  clients  on  a  routine  basis.  One  way  to  verify  the  income 
would  be  to  access  the  Department  of  Labor  Unemployment  System. 
Since  the  current  financial  summary  policy  is  to  complete  forms  as 
needed,  income  verification  would  be  a  good  measure  of  the  need  to 
update  financial  summary  forms. 

MVR  management  indicated  they  will  contact  the  Department  of 
Labor  and  Industry  to  determine  if  there  is  a  cost-effective  method  to 
randomly  check  incomes  levels. 

Recommendation  #9 

We  recommend  Montana  Vocational  Rehabilitation  Program 
management  establish  controls  to  verify  income  for  a  sample  of 
MVR  clients  on  a  routine  basis. 


Student  Financial  Aid  MVR  financial  policies  state  "No  training  services  in  institutions  of 

Documentation  Important  higher  education  shall  be  paid  for  with  funds  under  Title  1  unless 

maximum  efforts  have  been  made  to  secure  grant  assistance  in  whole 
or  in  part  from  other  sources  to  pay  for  such  training."  Policies 
require  a  client  to  apply  for  a  federal  PELL  education  grant  to  assist 
with  the  cost  of  training  prior  to  determining  the  amount  of  MVR 
funding  a  client  should  receive  for  education  expenses. 

Thirty-two  clients  in  our  file  sample  attended  universities,  colleges 
or  vocational  schools.  We  found  MVR  counselors  did  not  always 
require  documentation  the  clients  applied  for  or  received  financial 
aid.  We  noted  some  instances  where  counselors  requested  clients 
apply  for  financial  aid,  but  the  file  did  not  contain  documentation  to 
ensure  the  client  had  applied. 
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Verincation  of  Financial 
Aid  is  Needed 


We  contacted  the  schools  clients  in  our  sample  attended.  We 
verified  if  clients  applied  for  student  financial  aid  and  the  amount 
awarded  to  each.  According  to  information  received  from  the 
schools,  22  of  the  32  MVR  clients  attending  school  received  a  total 
of  $228,949  in  student  financial  aid  while  receiving  services  from 
MVR.  MVR  file  documentation  showed  16  clients  received 
financial  aid  for  a  total  of  $44,266.  Table  12  shows  the  results  of 
our  file  review  and  information  received  from  the  individual 
schools. 


Table  12 
Comparison  of  Student  Financial  Aid  Documentation 


Financial  Aid 
Status 

MVR 
Documentation 

School 
Documentation 

Received  Award 

16 

22 

Not  Eligible  For  Financial 
Aid 

5 

1 

Did  Not  Apply  For 
Financial  Aid 

7 

2 

Unable  to  Determine 

4 

6 

Declined  Award 

0 

1 

Financial  Aid  Not 
Applicable 

0 

1 

Total  Clients 

*32 

*33 

*  Totals  do  not  match  since  one  cUent  attended  two  schools. 

Source:       Compiled  by  the  Legislative  Audit  Division  from 
MVR  files  and  school  records. 


By  not  verifying  financial  aid,  MVR  may  have  paid  more  than 
necessary  for  tuition,  books,  and  school  supplies.  Without 
documentation  of  an  award  letter  verifying  the  client  applied  for 
financial  aid  and  the  amount  awarded,  MVR  does  not  know  whether 
the  client  is  appropriately  participating  in  their  rehabilitation  plan. 
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The  current  process  does  not  ensure  accurate  client  cost  participation 
for  educational  expenses. 

Additional  information  on  financial  aid  awards  is  available  directly 
firom  the  schools.  Financial  aid  officers  indicated  if  requested,  they 
could  send  MVR  copies  of  the  award  letters,  student  aid  reports,  or 
aid  statements  for  clients  attending  their  school. 


Program  Action:  Award 
Letters  Required 


MVR  now  has  policy  which  requires  the  financial  award  letter  to  be 
in  case  flies. 


Recommendatioilllflf 

We  recommend  the  Montana  Vocational  Rehabilitation  Program 
strengthen  management  controls  by  requiring  an  award  letter  or 
other  necessary  information  prior  to  paying  education  expenses. 


Case  File  Closure 


The  reasons  why  a  case  file  is  placed  in  a  closed  status  range  from 
client  ineligibility  to  a  client  successftilly  maintaining  a  job.  The 
following  were  the  closed  statuses  used  by  MVR  during  our  audit 
period. 


►  Status  08        MVR  determines  an  individual  ineligible  for  VR 

services. 

*■      Status  26        Closed  rehabilitated,  employed  successfully  for  at 
least  60  days. 

►  Status  28        Closed  after  an  IWRP  is  written,  or  closed  after  at 

least  one  service  outlined  in  the  IWRP  is 
provided,  but  before  the  client  is  employed. 

•■      Status  30        Closed  after  an  IWRP  is  written,  but  before  any 
of  the  services  outlined  in  the  IWRP  are 
provided. 

»•      Status  33        Closed,  client  finished  receiving  post-employment 
services  for  the  present  and  employment  is 
maintained. 
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►      Status  35        Client  finished  receiving  post-employment 

services  and  is  placed  back  in  referral  status  to 
begin  a  new  rehabilitation  program. 

»■      Status  37        Client  finished  receiving  post-employment 

services  and  the  file  is  closed  for  reasons  other 
than  employment  or  referral.  (The  client  is  not 
employed  and  will  not  receive  services  from 
MVR.) 

A  client's  case  can  be  closed  at  anytime  during  the  vocational 
rehabilitation  process.  For  example,  a  case  file  can  be  closed  when 
it  is  determined  the  individual  is  ineligible,  refuses  services,  does  not 
cooperate,  is  no  longer  available  for  services,  or  has  achieved  a 
successful  outcome. 


Clients  Notified  of  Closure 


According  to  MVR  policy,  the  counselor  must  send  a  client  written 
notification  signed  by  the  counselor  notifying  the  individual  of 
reasons  for  closure.  The  written  notice  informs  the  client  of  his/her 
right  to  appeal  the  decision.  It  is  also  the  counselor's  responsibility 
to  refer  individuals  to  other  agencies  or  programs  when  appropriate. 
Counselors  try  to  obtain  the  signature  of  a  client  when  the  case  is 
closed  with  or  without  a  successful  outcome.  In  many  cases  two  or 
more  letters  are  sent  to  the  individual,  with  at  least  one  being 
registered  mail  with  a  return  receipt  required.  These  letters  are 
designedio  inform  the  client  the  case  is  closed  and  try  to  obtain  a 
signature.  Policy  states  letters  sent  to  clients  achieving  a  successful 
outcome  require  the  client's  signature  and  a  copy  of  the  letter  is 
returned  to  MVR  as  evidence  of  the  client's  involvement  in  closure. 


Post-Employment  Services 
Provided 


After  a  client  has  achieved  what  is  considered  a  suitable  employment 
outcome,  MVR  can  provide  post-employment  services.  Post- 
employment  services  are  goods  and  services  needed  after  clients  are 
employed  and  are  necessary  to  enable  the  person  to  maintain,  regain, 
or  advance  in  employment.  Post-employment  purchases  include 
goods  and  services  such  as  an  eye  examination  and  glasses,  clothing 
for  work,  and  job  placement  services. 
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Occupation  at  Closure 


From  information  in  the  files,  we  identified  the  client's  occupation  at 
closure.  We  gathered  information  for  all  types  of  closures,  not  just 
those  employed  successfully  for  at  least  60  days. 


We  categorized  occupations  into  service,  clerical,  and  trade  pro- 
fessions. Examples  of  service  professions  include  individuals 
employed  as  a  waiter/waitress,  janitors,  or  licensed  practical  nurses. 
Clerical  positions  include  individuals  working  as  clerks,  book- 
keepers, or  secretaries.  The  trade  professions  include  individuals 
working  as  mechanics,  meat  cutters,  or  laborers.  Table  13  breaks 
down  the  number  of  clients  and  different  types  of  occupations  held 
upon  closure. 


Table  13 
Type  of  Occupation  at  Closure 


Type  of  Occupation 

Number  of  Clients 

Service 

35 

Clerical 

7 

Trade 

6 

Source:       Compiled  by  the  Legislative  Audit  Division  from 
MVR  records. 


Summary 


We  believe  implementing  the  recommendations  discussed  in  this 
chapter  would  increase  the  efficiency  and  effectiveness  of  MVR's 
delivery  of  services  to  eligible  clients.  Clarifying  and/or  writing 
policies,  providing  training,  and  reviewing  more  closed  files  would 
ensure  federal  regulations  are  followed  and  clients  are  treated 
consistently. 


Other  areas  of  increased  efficiency  and  effectiveness  pertain  to 
purchasing  goods  and  services  and  program  organizational  controls. 
These  subjects  are  discussed  in  the  Chapters  V  and  VI. 
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Introduction 


MVR  provides  counseling  and  guidance,  vocational  evaluation,  and 
vocational  placement  services  without  cost  to  an  eligible  person.  In 
addition,  MVR  may  pay  the  cost  of  other  vocational  rehabilitation 
services  for  an  eligible  person  in  financial  need.  During  the 
purchasing  audit  of  MVR,  we  examined  the  process  to  obtain  goods 
and  services  for  MVR  clients.  We  did  not  review  the  purchasing 
procedures  for  department  goods  and  services. 


The  objective  of  the  purchasing  audit  was  to  determine  if  purchasing 
procedures  are  followed  which  ensure  the  receipt  of  goods  and 
services  by  MVR  clients.  We  answer  the  following  questions  in  this 
chapter: 

1 .  What  types  of  goods  and  services  were  provided  to  MVR  clients 
during  fiscal  year  1994-95? 

2.  How  does  the  department  ensure  vendors  provide  agreed  upon 
goods  and  services  to  MVR  clients  and  purchasing  procedures 
are  followed? 


4. 


Are  the  vocational  evaluation  tests  purchased  for  MVR  clients 
providing  counselors  with  information  needed  to  make 
reasonable  decisions  concerning  the  client's  vocational  aptitude? 

How  much  did  the  department  expend  on  MVR  goods  and 
services  for  fiscal  year  1994-95? 


Types  of  Goods  and 
Services  Provided 


Our  file  review  included  gathering  information  as  to  the  types  of 
goods  and  services  purchased  by  MVR  for  clients.  For  example, 
medical  and/or  psychological  tests  and  reports  are  purchased  for 
most  clients.  Many  clients  also  receive  vocational  assistance. 


Table  14  summarizes  the  types  of  goods  and  services  purchased  and 
the  number  of  clients  receiving  the  goods  and/or  services  for  the  150 
MVR  files  reviewed. 
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Table  14 
Tvpes  of  Goods  and  Services  Purchased  for  MVR  Clients 

Types  of  Goods  and  Services 

ISumber 
orchenls 

Medical/Psychological  Reports  &  Tests 

108 

Vocational  Assistance 

79 

College  or  University  Education 

20 

Vocational  Education 

13 

Adult  Basic  Education 

2 

Fuel/Vehicle  Repair 

24 

Clothing 

16 

Travel/Moving 

11 

Rent 

10 

Food 

10 

Vehicle  Insurance 

5 

Day  care,  etc. 

3 

Other  (eye  exams,  glasses,  orthotics) 

24 

Source:          Compiled  by  the  Legislative  Audit  Division 
from  MVR  records. 

Types  of  Vendors 
Providing  Goods  and 
Services 


MVR  purchases  goods  and  services  from  a  variety  of  vendors,  such 
as  medical  doaors  and  psychologists,  educational  facilities,  clothing 
stores,  insurance  companies,  day  care  providers,  privately  run 
rehabilitation  programs,  and  other  vendors  necessary  for  client 
rehabilitation. 


Purchasing  Procedures 


All  goods  and  services  for  the  client  are  to  be  identified  in  the 
IWRP.  The  IWRP  documents  the  vendor,  estimated  cost,  and  time 
period  in  which  goods  and  services  are  anticipated  to  be  purchased. 


The  MVR  purchasing  process  changed  in  July  1996  when  the 
program  converted  to  a  new  computer  system.  Previously,  for  every 
purchase,  a  counselor  in  the  field  office  signed  and  sent  an 
authorization  for  the  services  to  the  central  office  in  Helena.  A 
purchase  order/billing  invoice  was  then  generated  and  sent  to  the 
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counselor  and  vendor.  Under  the  new  system,  the  counselor 
generates  an  authorization  and  purchase  order/billing  invoice  in  the 
regional  office  and  sends  the  forms  directly  to  the  vendor.  After  the 
vendor  provides  the  good  or  service  for  the  client,  the  vendor  signs 
the  billing  invoice  for  assurance  the  good  or  service  was  provided. 
The  signed  invoice  is  sent  to  the  counselor  for  his/her  signature  to 
confirm  receipt  of  the  good  or  service  and  the  bill  is  forwarded  to 
Helena  for  payment. 


Conclusion:  Purchasing 
Procedures  Followed  and 
Ensure  Receipt  of  Goods 
and  Services 


Based  on  our  review  of  MVR  files,  purchase  authorization  and 
purchase  orders  were  used  appropriately  99.7  percent  of  the  time 
(1 137  out  of  1 140  purchases).  We  did  not  find  any  concerns  that 
clients  did  not  receive  goods  and  services  purchased  by  MVR. 


Since  MVR  policy  is  for  the  vendor  and  counselor  to  sign  the 
purchase  order  showing  the  transaction  is  complete,  we  documented 
when  there  was  no  counselor  signature  or  no  vendor  signature.  Of 
the  1 140  purchases,  14  were  not  signed  by  the  counselor 
(1.2  percent)  and  16  had  no  vendor  signature  (1.4  percent).  We  also 
looked  to  see  if  the  client  signed  for  purchases  when  the  money  went 
directly  to  them  instead  of  the  vendor.  Since  the  client  must  sign  the 
check  to  cash  it,  MVR  policy  does  not  require  client  signature  on  the 
purchase  order  invoice. 

We  conclude  MVR  purchasing  procedures  ensure  receipt  of 
purchased  goods  and  services  and  staff  follow  written  purchasing 
policies  and  procedures. 


Services  Provided  by 
Community  Rehabilitation 
Programs 


MVR  purchases  many  types  of  services  from  19  certified 
Community  Rehabilitation  Program  (CRP)  providers  around  the 
state.  The  CRP's  provide  direa,  or  facilitate  a  provision  of, 
vocational  rehabilitation  services  to  individuals  with  disabilities. 
The  CRP's  provide  one  or  more  of  four  types  of  services  defined  in 
Table  15. 
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Table  15 
Services  ProYided  by  Community  Rehabilitation  Program 


Service 

Description 

Vocational  Evaluation 

A  comprehensive  process 
that  uses  work,  eimer  real  or 
simulated,  for  assessment  or 
vocational  exploration. 

Sheltered  Work  Adjustment 
Training  (WAT)  Services 

A  time  limited  training 
program  which  assists 
persons  toward  job 
placement  utilizmg  real  or 
simulated  work  in  non- 
integrated  settings  (the  CRP 
is  the  employer). 

Transitional  Employment 
Services  (ItS) 

Services  provided  in  one-to- 
one  or  group  settings  to 
provide  training  at  the 
employment  site. 

Placement  Services 

Services  provided  to  prepare 
for  job  finding,  employer 
contact  and  negotiation,  job 
accommodation,  and  other 
job  related  services  (except 
skill  training  at  the  job  site). 

Source:       Compiled  by  the  Legislative  Audit  Division  from 
MVR  records. 


CRP  Certification 


DPHHS  requires  CRPs  providing  services  to  MVR  clients  to  be 
accredited  by  the  Commission  of  Accreditation  of  Rehabilitation 
FacUities  (CARF)  or  the  National  Accrediting  Council  (NAC). 
DPHHS  will  certify  a  provider  that  is  not  CARF  or  NAC  accredited 
if  the  provider  is  certified  by  a  state  or  federal  agency  with  which 
DPHHS  has  a  cooperative  agreement  concerning  the  coordinated 
delivery  of  services. 


All  accredited  CRPs  are  provided  an  accreditation  standards  manual 
to  help  ensure  the  delivery  of  quality  services  to  people  with 
disabilities.  Standards  are  established  for  every  program  or  service  a 
CRP  provides  to  MVR  clients  including  intake,  orientation, 
individual  planning,  referral,  exit/discharge,  and  follow-up.  In 
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addition,  standards  address  facility  organization  such  as  consumer- 
based  planning,  organizational  structure  and  ownership, 
organizational  management,  information  management,  program 
evaluation,  assessment  of  program  quality,  fiscal  management,  and 
human  resources. 


Conclusion:  CRP 
Vocational  Evaluations  Aid 
Counselors  in  Decision 
Making 


When  needed,  MVR  purchases  vocational  evaluations  for  clients  to 
determine  their  vocational  aptitude.  We  reviewed  the  vocational 
evaluation  process.  The  evaluations  consist  of  nationally  accepted 
tests  administered  and  interpreted  by  qualified  CRP  staff.  Test 
results  are  summarized  in  a  report  provided  to  the  MVR  counselor 
and  discussed  with  the  client. 


Based  on  our  review  of  documentation  and  interviews  with  CRP  staff 
and  MVR  counselors,  we  determined  the  vocational  evaluation  tests 
given  to  MVR  clients  provide  counselors  with  necessary 
information.  The  tests  are  used  to  make  reasonable  decisions 
concerning  the  client's  vocational  aptitude.  The  vocational 
evaluations  help  determine  which  services  assist  the  client  in 
attaining  an  employment  outcome. 


Fee  Limits 


During  our  audit  work,  we  reviewed  existing  fee  limits  of  goods  and 
services  purchased  for  MVR  clients  and  procedures  for  establishing 
new  limits  on  the  amount  to  be  paid.  The  following  sections 
describe  the  current  MVR  fee  structure. 


Medical  Services  Fee 
Limits  are  Established  in 
Policy 


Medicaid  reimbursement  rates  are  used  for  in-hospital  medical 
services  MVR  may  purchase.  All  other  medical  services  for 
providers,  other  than  hospitals,  are  established  in  the  MVR  fee 
schedule  included  in  the  MVR  policy  manual.  Medical  services  such 
as  physical  examinations,  eye  examinations,  dental  examinations, 
and  psychological  or  psychiatric  examinations  have  limits  as  to  the 
amount  MVR  pays. 
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CRP  Fee  Limits  Increased  MVR  approves  fee  limits  for  all  services  for  each  CRP  provider. 

on  Percentage  Basis  We  reviewed  the  procedures  for  establishing  CRP  fee  limits  and 

interviewed  counselor  supervisors  in  each  district  office.  As  CRP 
liaisons,  counselor  supervisors  act  as  an  agency  representative  for 
the  CRPs  in  their  area.  The  liaisons  write  a  cooperative  agreement 
between  each  CRP  and  MVR.  The  agreement  outlines  for  each 
CRP,  their  proposed  services  available,  expected  number  of  MVR 
clients,  and  incorporates  the  requested  fees  for  each  service.  The 
agreement  with  the  requested  fees  is  sent  to  Helena  for  approval. 

Initial  CRP  fee  limits  were  established  based  on  cost  data.  Since 
then,  limits  are  modified  on  a  percentage  basis  every  year  based  on 
increases  approved  by  the  legislature.  MVR  reviews  the  providers' 
requested  fee  changes  in  conjunction  with  the  legislative  increase  to 
establish  fee  limits.  While  costs  change  over  time,  these  costs  are 
not  necessarily  based  on  a  designated  percentage  increase.  For  the 
1997  biennium,  the  legislature  approved  a  CRP  provider  rate 
increase  of  $1  million  over  the  biennium. 

CRP  Fees  Vary  Our  review  of  CRP  fees  shows  variation  in  the  established  limits  for 

the  same  services.  For  example,  in  1995  vocational  evaluation  fee 
limits  ranged  from  $681  to  $998.  Some  providers  charged  a 
processing  fee  for  the  evaluations  ranging  from  $54  to  $91 .  (We 
included  the  processing  fee  in  the  evaluation  fee  ranges  to  show 
actual  fees  charged  for  that  service.)  Placement  services  fees  ranged 
from  $574  up  to  $1,435.  Since  each  region  establishes  fee  limits  for 
providers  in  that  region,  there  is  no  statewide  review  for  fee 
consistency  or  review  of  costs  to  determine  reasonable  fee  limits. 


Cost  Basis  in  Fee  Setting  The  CARP  manual's  fiscal  management  standard  indicates  charges 

for  services  and  products  should  be  based  on  a  knowledge  of  both 
direct  and  indirea  costs.  The  National  Association  of  State 
Purchasing  Officials  recommends  requiring  cost  or  price  analysis  for 
negotiated  prices.  The  International  City/County  Management 
Association  book  on  service  contracts  recommends,  for  social 
service  purchases,  the  offeror  list  all  costs  associated  with  the  service 
including  administrative  and  overhead  costs,  operating  costs, 
equipment  costs,  and  any  other  costs  associated  with  service 
delivery. 
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MVR  issued  a  procedural  memo  in  line  with  these  criteria  for  new 
services  or  new  providers.  MVR  CRP  liaisons  are  to  include  a 
review  of  the  provider's  direct  and  indirect  costs  when  establishing 
fees  for  new  services  or  new  providers.  However,  there  is  no  formal 
policy  requiring  a  review  of  existing  CRP  providers'  actual  costs  to 
modify  fee  limits. 


Accommodation  Charges 
Added  to  Total  Fee 


Other  costs  and  requirements  impact  MVR's  review  of  fees.  The 
Americans  with  Disabilities  Act  (ADA)  requires  accommodation 
needs  be  included  as  part  of  the  cost  of  doing  business. 
Accommodation  needs  include  goods  or  services  used  to  aide  a 
person  with  a  disability,  such  as  a  deaf  interpreter.  MVR  told  the 
CRPs  to  add  an  accommodation  charge  into  all  their  fees  to  cover 
the  cost  of  necessary  accommodations.  This  resulted  in  the  CRPs 
increasing  their  fees.  During  our  file  review  we  noted  one  CRP 
facility  adds  two  percent  onto  all  services.  Not  all  CRPs  itemize  the 
accommodation  fee  on  the  billing  invoices.  Since  MVR  does  not 
review  these  indirect  costs,  MVR  does  not  know  if  the  fees  charged 
are  a  reasonable  or  fair  representation  of  the  cost  of  these  services. 


Federal  requirements  establish  principles  to  be  applied  in 
establishing  the  allowability  or  unallowability  of  certain  items  of 
cost.  The  Office  of  Management  and  Budget  (0MB)  Circulars  A-87 
and  A-122  establish  cost  principles  which  disallow  charges  for 
purchases  which  cannot  be  predicted  with  any  assurance  of 
happening. 


MVR  Should  Review  Costs 


MVR  has  policies  and  procedures  for  establishing  fees  for  new 
providers  or  services.  MVR  should  develop  a  policy  to  review  cost 
information  for  all  providers  and  services  to  establish  reasonable  fees 
and  ensure  compliance  with  both  ADA  and  the  0MB  circulars. 
Reasonable  fee  limits  are  based  on  all  costs  associated  with  a  service 
including  direct  and  indirect  costs.  To  be  cost-effective,  once 
reasonable  fee  limits  are  established  MVR  may  then  be  able  to  select 
a  sample  of  providers  for  routine  follow-up  review  since  MVR 
modifies  CRP  fee  limits  aimually. 


Page  56 


Program  Action:  Fee 
Limits  Being  Reviewed 


Chapter  V  -  Purchasing  Process 


MVR  is  working  with  DPHHS's  internal  auditors  to  complete  a 
preliminary  review  of  CRP  revenues  and  expenditures.  The  internal 
audit  group  will  devise  a  method  to  review  fee  limits  during  their 
routine  audits  of  major  providers. 


Recommendation  #11 

We  recommend  Montana  Vocational  Rehabilitation  Program 
management: 

A.  Establish  a  policy  to  review  cost  information  including  direct 
and  indirect  cosis  for  all  providers  and  services. 

B.  Develop  a  process  to  routinely  review  fee  Umits  to  establish 
reasonable  fees  and  ensure  compliance  with  OMB  Circulars 
A-87  and  A-122  and  ADA  requirements. 


Management  Informa- 
tion on  Goods  and 
Services  Purchased  for 
MVR  Clients 


We  asked  central  office  staff  in  Helena  and  regional  administrators 
about  expenditures  for  specific  types  of  goods  and  services 
purchased  for  MVR  clients.  No  one  was  able  to  identify 
expenditures  by  type  of  goods  or  services  purchased.  MVR  tracks 
most  budgetary  information  by  type  of  funding  rather  than  type  of 
purchase. 


MVR  Budget  Process 
Should  be  More  Detailed 


We  then  reviewed  the  MVR  budgeting  process.  MVR  establishes  an 
initial  budget  for  each  region  based  on  number  of  clients  served. 
Monthly  budget  information  is  communicated  between  the  regional 
offices  and  the  central  office  in  Helena.  Quarterly,  the  budget  is 
redistributed  based  on  justification  provided  by  each  regional 
administrator.  Redistribution  is  based  on  the  various  funding 
sources  rather  than  analysis  of  specific  types  of  goods  and  services 
purchased. 


The  purpose  of  budgets  and  budgeting  is  to  help  decision  makers 
determine  which  activities  are  most  desirable  and  to  allocate 
resources  among  them  as  efficiently  as  possible.  Detailed 
management  information  of  client  purchases  allows  managers  to 
better  administer  then-  programs,  especially  with  equitable 
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distribution  of  the  budget.  Information  on  the  types  of  purchases 
would  provide  a  better  indicator  of  client  needs  in  each  region. 


File  Review  Results  of 
Purchased  Goods  and 
Services 


As  a  result  of  MVR  not  compiling  budgetary  information  by  type  of 
goods  and  services,  staff  were  unable  to  readily  identify  how  much 
was  spent  on  any  one  type  of  good  or  service  for  clients.  As  an 
example  of  the  type  of  information  which  might  help  manage 
resources,  we  categorized  purchases  in  our  sample  of  150  MVR  files 
(of  which  only  122  clients  had  purchases)  into  several  areas 
including: 


Job-related 

School-related 

Medical  and  psychological 

Vocational  evaluation 

Maintenance 

Transportation 

Computer  equipment  and  supplies. 


We  then  added  the  purchases  for  each  category  for  each  of  the  four 
regional  offices.  We  found  one  region  expended  significantly  more 
than  the  other  three  regions  in  most  categories.  MVR  management 
could  use  this  type  of  information  to  analyze  reasons  why  one  region 
spends  more  than  another  or  to  determine  if  there  is  a  correlation 
between  services  purchased  and  successful  job  placement.  Table  16 
shows  the  results  of  our  file  review. 
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Table  16 
MVR  File  Review  Results  of  Goods  and  Services  Purchased  for  Clients 

Item 

Region 
Number 

#of 
Clients 

#of 
Items 

Cost 

Avg. 

Cost 

Per 

Client 

Job-Related  Purchases  including:  Job  Club, 
Coaching.  Development.  Placement,  Readiness, 
Seeking,  Follow-Along,  Sheltered  and  Supported 
Employment,  OJT,  Work  Adjustment  Training, 
Work  Assistance,  and  Transitional  Employment. 

1 
2 
3 
4 
Total: 

12 

5 

14 

11 

42 

68 

22 

108 

283 

$  29,314 
13,182 
47,640 

*  27.056 
$117,192 

$2,443 
2,636 
3,403 
2.460 

$  2,790 

School-Related  Purchases  including:  Tuition, 
Fees,  Books,  SuppUes,  Tutoring,  Exams  and 
Insurance. 

1 

2 

3 

4 

Total: 

7 

5 

13 

10 

35 

41 

58 
147 

37 
283 

$  25,358 
23,258 
79.348 

^  23.035 
$150,999 

$  3,623 
4,652 
6,104 
2.304 

$4,448 

Medical  and  Psychological  Purchases  including: 
Records,  Exams,  Evaluations,  Counseling, 
Physicals,  Surgery,  Eye  Glasses.  Dental  Work, 
Medication.  Health  Club  Membership. 

1 
2 
3 
4 
Total: 

18 

27 
27 
22 
95 

54 
55 
70 
65 
244 

$    6.130 
7.981 
4.082 

$  26,547 

$     341 
296 
159 
363 

$     279 

Vocational-Evaluation 

1 

2 

3 

4 

Total: 

9 
9 

8 

? 
34 

9 
9 

8 

8 
34 

$    4,806 
6,855 
5,007 

*  5.176 
$21,844 

$     534 
762 
626 
647 

$     642 

Maintenance  including:  Clothing.  Lunches, 
Motels,  Living  Expenses,  Rent,  Rent  Deposit, 
Day  Care,  Um  Cuts,  and  Emergency 
Assistance. 

1 
2 
3 
4 
Total: 

4 

7 

10 

7 
28 

12 
15 
46 
14 

87 

$    1,844 

1,896 

11,485 

1.275 

$  16,500 

$     461 

271 

1.149 

182 

$     589 

Transportation  including:  Gas,  Vehicle 
Insurance,  Bus  Passes,  Parking  Permits,  Vehicle 
Repairs  and  Maintenance,  Partial  Payment  of 
Van,  and  Moving  Truck  Rental. 

1 
2 
3 
4 
Total: 

6 

6 

12 

10 
34 

41 

14 

102 

47 
204 

$     1,596 

768 

7,281 

5.044 

$  14,689 

$     266 
128 
607 
504 

$     432 

Computer  Equipment  and  Supplies 

1 

4 
Total: 

1 

2 
3 

3 
2 
5 

$        655 

.    2.591 
$     4,246 

$     655 
1.796 

$  1.415 

Grand  Total: 

1 
2 
3 
4 
Total: 

28 
30 
32 
32 
122 

228 
173 
481 
258 
1140 

69,703 

53,940 

154,843 

*  73.531 
$352,027 

$  2.489 
1,798 
4.839 
2.298 

$  2,885 

Source:       Compiled  by  the  Legislative  Audi 

t  Division  f 

romMVI 

I  records. 
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Analysis  of  Purchases  The  management  information  reports  currently  used  by  MVR  are 

Important  primarily  based  on  federal  reporting  requirements  which  focus  on 

types  of  funding  rather  than  types  of  purchases.  Budget  distribution 
and  re-distribution  are  based  on  types  of  fiinding.  MVR  could 
improve  the  information  used  to  manage  resources. 

MVR  is  working  to  identify  necessary  purchasing  information  and 
will  generate  management  reports. 

Recommendation  ni 

We  recommend  Montana  Vocational  Rehabilitation  Program 
management  identify  and  generate  information  to  help  manage 
program  resources  including  budgeting  and  analysis  of 
purchasing  in  each  r^on. 
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Introduction 


Organizational  Controls 


Goals  and  Objectives 


Interviews  with  program  management  and  staff,  and  review  of 
program  management  information  provided  us  with  the  basis  to 
answer  the  last  three  objectives  of  this  audit:  1)  What  are  the 
outcomes  of  the  MVR  Program?;  2)  Are  organization  controls 
consistent  with  the  intent  of  the  MVR  Program?;  and  3)  How  does 
MVR  monitor  its  success  and/or  effectiveness?  The  following 
discusses  results  of  the  audit  work  and  our  recommendations  for 
improvement. 

The  role  of  management  entails  directing  the  use  of  limited  resources 
in  the  most  efficient  and  effective  maimer  while  fiilfilling  the 
statutory  purposes  of  a  program.  One  way  to  direct  program  staff  is 
by  the  development  of  sound  organizational  controls.  Organiza- 
tional controls  include  measurable  standards  and  systems.  Examples 
of  measurable  standards  include  goals,  objectives,  policies,  and 
procedures.  The  measurement  system  in  organizational  controls  is 
primarily  management  information.  The  following  discusses  various 
components  of  the  organizational  controls  at  MVR. 

Documenting  relevant,  verifiable  goals  and  measurable  objectives  is 
fundamental  to  establishing  measurable  standards  for  a  program. 
Goals  should  relate  to  the  purpose  and  mission  of  a  program, 
correspond  to  the  statutory  intent,  and  provide  a  clear  direction  for 
action.  Setting  goals  requires  developing  questions  to  be  answered 
and  establishing  objectives  which  identify  specific  levels  of 
achievement  toward  that  goal.  Objectives  relate  directly  to  program 
goals  and  should  describe  specific  and  measurable  targets,  results,  or 
outcomes.  Defined  objeaives  are  specific  levels  of  achievement. 
Objectives  should  be  result  oriented,  attainable,  and  allow 
management  to  track  responsibilities  and  accomplishments  within  a 
program. 

MVR's  current  program  goals  and  objectives  are  based  primarily 
upon  federal  funding,  originating  from  historic  production  figures 
(number  of  clients  at  a  specific  status).  MVR  management  in  turn 
uses  these  numbers  as  goals  to  determine  if  the  program  is  cost- 
effective  and  evaluates  staff  based  upon  these  production  figures. 
MVR  has  also  developed  value-based  goal  statements  which  support 
federal  regulations  and  support  basic  vocational  rehabilitation 
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philosophy.  An  example  of  one  of  these  statements  is  "MVR  values 
an  enhanced  and  expanded  service  delivery  that  is  responsive  to  the 
essential  needs,  dignity,  and  rights  of  individuals  with 
disabilities  .  .  .." 

Using  historic  production  figures  as  goals  and  objectives  provides  a 
level  of  achievement  by  staff,  yet  does  not  provide  ways  for  staff  or 
management  to  achieve  the  numbers  or  to  evaluate  compliance  with 
the  program's  mission. 

Our  review  of  the  MVR  Program  identified  areas  where  MVR  could 
strengthen  organizational  controls  of  the  program,  specifically  goals 
and  objectives.  The  following  are  examples  of  goals  and  objectives 
MVR  could  establish: 

Goal  Increase  number  of  individuals  applying  for  MVR 

services. 
Objectives  Develop  a  statewide  advertising  campaign. 

Evaluate  the  application  process  with  emphasis  on 

streamlining. 

Goal  MVR  will  ensure  the  maximum  and  efficient  use  of 

MVR  benefits  and  resources. 

Objectives  Ensure  equal,  consistent  treatment  of  clients. 

Develop  policies  and  procedures  to  help  ensure 
client  financial  contributions  are  equitably 
considered  and  recovered. 

By  expanding  and  revising  already  existing  goals  and  objectives, 
MVR  could  strengthen  the  program's  mission  and  purpose. 

MVR  management  indicated  their  Strategic  Plan  will  contribute  to 
the  development  of  more  tangible  measurable  goals  and  objeaives. 
Federal  regulations  require  a  strategic  plan  be  written  each  year  to 
supplement  the  State  Plan  which  is  required  every  three  years. 
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Recommendation  #t^ 

We  recommend  Montana  Vocational  Rehabilitation  Program 
management: 

A.  Revise  goals  that  relate  more  to  the  purpose  and  mission  of 
the  program. 

B.  Establish  measurable  objectives  which  relate  directly  to 
goals  and  allow  for  attainable  results  or  outcomes. 


Management  Information 
System 


Current  Management 
Information  Reports 


A  management  information  system  is  a  data  network  designed  to 
provide  managers  with  timely  and  useful  information  for  effective 
planning  and  control.  Management  information  helps  individuals 
make  decisions.    Management  information  is  a  way  to  monitor  and 
evaluate  program  operations  and  measure  program  outcomes  or 
results.  Managers  must  analyze  the  information  and  program 
outcomes,  compare  them  with  expected  goals,  decide  the  importance 
of  any  differences,  and  make  necessary  changes.  When  management 
information  is  available  to  individuals  outside  of  an  organization,  it 
provides  an  overall  understanding  of  the  program  and  its  outcomes. 
Management  information,  as  a  measure  of  program  performance, 
holds  public  officials  accountable  for  performance  on  service 
quantity,  quality,  and  efficiency,  as  well  as  guide  improvements  in 
public  service. 

MVR  generates  more  than  35  management  reports  with  program 
information.  These  reports  are  generated  either  biweekly,  monthly, 
quarterly,  or  at  federal  fiscal  year  end.  Most  of  these  reports  track 
information  required  to  obtain  federal  funding.  Consequently,  MVR 
reports  are  primarily  based  on  federal  reporting  requirements  which 
mainly  focus  on  types  of  funding  rather  than  overall  program 
activities.  For  example,  when  asked  how  many  vocational  evalu- 
ations were  purchased,  MVR  could  identify  only  the  number  of 
evaluations  purchased  with  federal  Section  1 10  funding;  they  could 
not  produce  the  total  number  of  vocational  evaluations  purchased  for 
MVR  clients. 
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Other  MVR  reports  appeared  to  show  overall  program  details,  yet 
were  found  to  have  limited  use  in  analyzing  program  operations. 
For  example,  there  is  a  report  with  the  cost  and  average  cost  of 
services  purchased  for  individuals  by  disability  category.  The  report 
identifies  purchases  of  services  by  disability  type  which  may  be 
useful  information  in  managing  program  resources.  However,  the 
computer  is  programmed  such  that  for  each  client,  the  most  recent 
service  is  entered  into  a  disability  code,  but  then  all  subsequent 
services  are  added  to  a  subtotal  for  either  severely  disabled  or  other 
disabilities  category.  Consequently,  MVR  management  cannot 
determine  the  actual  cost  or  average  cost  of  any  disability  category. 

Some  Management  Reports  As  a  result  of  less  than  complete  data  and  in  some  cases  inaccurate 

not  Useful  data  on  MVR's  management  reports,  staff  do  not  fully  use  some 

reports.  For  example,  there  is  a  report  used  by  administrators  and 
counselors  which  tracks  the  status  of  each  client.  The  old  computer 
system  did  not  accurately  track  length  of  time  a  client  was  in  a 
status;  therefore,  the  information  could  not  be  used  to  accurately 
monitor  caseload  information. 

Without  useful  information,  program  managers  cannot  make 
decisions  on  whether  expectations  are  met  or  if  changes  are 
necessary  to  improve  program  operations.  By  not  having  access  to 
useful  management  information,  it  is  difficult  to  measure 
achievement  of  goals  and  objectives  and  measure  the  efficiency  and 
effectiveness  of  program  operations. 

MVR  believes  their  new  computer  system  will  help  address  the  need 
to  access  necessary  information. 

We  recommend  Montana  Vocational  Rehabilitation  Program 
management: 

A.  Identify  wiiat  information  would  be  useful  to  help  manage 
program  operations  and  measure  program  outcomes. 

B.  Devdop  that  information  into  useful  managemait  reports. 
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Success/Effectiveness  of 
MVR 


To  effectively  evaluate  the  program's  success  or  effectiveness, 
management  should  establish  a  process  or  system  to  track 
management  responsibilities  and  evaluate  program  accomplishments. 
According  to  standards  of  the  Commission  of  Accreditation  of 
Rehabilitation  Facilities  (CARF),  a  system  of  information  collection, 
dissemination,  and  utilization  should  be  accurate,  communicated  to 
appropriate  personnel,  and  relevant  to  an  organization.  The 
information  must  be  timely  and  reviewed  periodically  for  accuracy 
and  relevancy.  Finally,  results  of  the  analysis  should  be  used  at  all 
levels  of  decision  making  for  continuous  improvement.  DPHHS  has 
not  established  sufficient  controls  to  evaluate  the  MVR  Program's 
success  or  effectiveness.  One  way  to  begin  an  evaluation  of  a 
program  is  by  setting  performance  measurements. 


Performance 
Measurements 


Performance  measurements  allow  for  a  gauge  of  the  program's 
accomplishments.  They  help  improve  program  goals  and  objectives 
by  evaluating  policies  and  procedures  and  help  measure  the  results 
against  an  agreed  upon  standard.  Good  performance  measurements 
are  relevant  to  an  organization's  mission  and  goals,  and  are  reliable 
enough  to  produce  accurate  and  verifiable  information.  Overall, 
performance  measurements  evaluate,  fine  tune,  and  improve 
program  operations  and  help  management  plan  for  the  future. 


Staff  believe  they  are  successful  if  they  come  close  to  or  meet 
numbers  of  successful  placements  determined  by  management. 
According  to  management,  these  numbers  are  driven  by  historic 
figures.  Because  MVR  is  primarily  driven  by  its  funding  source, 
management  has  not  developed  its  own  performance  measures. 


Assessing  Performance 


Assessing  the  performance  of  a  program  includes  an  evaluation  not 
only  of  its  resources  and  its  process,  but  also  a  measure  of  its 
outcomes.  Outcomes  are  important  to  a  program  because  they  focus 
on  a  necessary  level  of  activity,  for  example  the  number  of  clients 
receiving  higher  education  as  a  means  to  obtain  a  successful 
occupational  placement. 


CARF  standards  also  recommend  a  program  measure  its  outcomes  or 
results  of  services  provided  and  regularly  measure  its  progress.  An 
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evaluation  of  outcomes  assesses  program  impact,  success,  and 
effeaiveness.  Outcome  indicators  are  important  performance 
measures  because  they  show  whether  or  not  expected  results  are 
achieved.  A  measurement  of  quality  to  determine  MVR  effective- 
ness could  include  courtesy,  reliability,  accuracy,  competence, 
responsiveness,  and  completeness  associated  with  services  provided. 
The  following  are  two  examples  in  which  MVR  could  expand  their 
evaluation  and  measurement  of  program  achievements. 

Customer  Satisfaction  MVR  measures  the  quality  of  customer  services  by  sampling  a 

random  selection  of  clients.  These  clients  are  asked  to  complete  a 
customer  satisfaction  survey.  The  survey  is  designed  to  give  clients 
the  opportunity  to  provide  information  on  how  they  perceive  the 
MVR  Program.  The  survey  results  are  compiled  by  staff  in  Helena 
and  distributed  to  each  region  for  their  review.  In  addition,  two 
regional  offices  have  clients  complete  the  "How  Did  We  Do?"  form. 
These  questions  are  similar  to  the  random  customer  satisfaction 
survey.  The  two  administrators  look  at  the  competed  forms  to 
identify  any  concerns.  MVR  staff  has  received  training  in  customer 
service. 

Despite  what  MVR  is  doing  to  evaluate  its  customer  service,  it  is  not 
completing  comparison  evaluations  of  responses  over  time.  For 
example,  MVR  could  expand  its  assessment  of  customer  surveys  to 
consider  what  improvements  or  changes  should  be  made  to  program 
operations  from  a  statewide  perspective  and  evaluate  the 
implemented  improvements  over  an  established  time  frame.  MVR 
should  also  consider  additional  elements  when  evaluating  the  overall 
success  and/or  effectiveness  of  a  program.  For  example,  MVR 
could  compile  information  to  determine  if  there  is  a  correlation 
between  the  types  of  disabilities,  services,  and  placements.  They 
could  compile  statistics  to  identify  critical  areas  where  resources  are 
expended,  which  would  ultimately  help  project  future  funding  needs, 
staffing  needs,  and  service  needs.  This  information  could  help 
counselors  develop  services  or  curriculum. 
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Long-term  Effects  of 
Successful  Placements 


Interviews  with  staff  disclosed  success  is  defined  as  60-day  job 
placements  for  clients.  This  is  a  minimum  federal  requirement. 
Long-term  effect  of  successful  placement  is  an  area  MVR  could 
assess.  Audit  work  included  an  evaluation  of  successfully  closed 
client  files.  We  gathered  information  of  those  clients  in  our  random 
sample  successfully  employed  for  60  days.  We  determined  if  the 
clients  continued  to  be  employed  as  of  the  first  quarter  of  1996.  We 
compiled  information  from  the  state  Unemployment  Insurance 
system.  We  were  unable  to  verify  employment  of  those  clients 
successfully  closed  as  homemakers,  sheltered  workshop  employees, 
or  self-employed  clients.  We  found  20  of  24  (83  percent)  of 
successful  closures  continued  to  be  employed  as  of  the  first  quarter 
of  1996.  In  addition,  eight  other  individuals  received  post 
employment  services  from  MVR  and  were  employed  for  the  60-day 
requirement.  Of  those  eight,  five  continued  to  be  employed  as  of  the 
first  quarter  of  1996.  These  statistics  indicate  successful  placement 
for  MVR. 


Summary 


It  is  important  for  MVR  to  involve  the  appropriate  program  staff  in 
the  development  of  performance  measures.  Involving  staff  at  all 
levels  ensures  performance  measures  reflect  what  can  be  achieved 
and  encourages  staff  to  take  ownership  in  the  measures.  MVR 
should  also  inform  all  staff  of  performance  measures.  In  addition, 
MVR  should  establish  a  system  for  timely  review  of  the  performance 
measures  to  help  ensure  they  address  the  changing  needs  of  the 
program. 


iiiommiidation  #15 
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Dear  Jim: 

Enclosed  is  a  copy  of  your  final  report  on  the  audit  of  the  Montana  Vocational  Rehabilitation 
Program  and  our  responses  to  your  recommendations.    I  understand  that  the  Legislative 
Audit  Committee  will  be  meeting  on  June  19  to  review  this  report. 

While  audits  are  difficult,  I  certainly  appreciate  the  professionalism  and  expertise  of  your 
staff  who  have  worked  with  us  over  these  past  months.    They  worked  tremendously  hard  and 
learned  a  lot  about  the  Montana  Vocational  Rehabilitation  Program.    We  really  did  enjoy  our 
dialog  with  them  and  we  appreciated  their  willingness  to  travel  to  our  field  offices. 

You  will  note  in  our  responses  that  many  of  the  recommendations  have  already  been  acted  on 
in  terms  of  implementation.    Please  feel  free  to  contact  me  if  you  or  your  staff  have  any 
questions  with  regard  to  our  responses. 


^^.(X\LmX 


Joe  A.  Mathews,  Administrator 

Disability  Services  Division 

Department  of  Public  Health  and  Human  Services 
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Recommendation  ffl 

We  recommend  Montana  Vocational  Rehabilitation  (MVR)  Program  management: 

A.  Complete  supervisory  reviews  of  a  sample  of  closed  files,  including  those  closed 
because  the  applicant  was  determined  ineligible. 

B.  Train  staff  on  eligibility  criteria  to  ensure  consistent  application  of  federal 
requirements  and  proper  documentation. 

MVR  Response:  We  have  already  begun  the  development  of  policy  that  defines  the 
consultation  and  mentoring  levels  of  counselors,  peers,  and  supervisors  in  the  areas  of  case 
closure  rationale  and  eligibility  determination.    In  the  course  of  the  next  federal  fiscal  year 
which  begins  October  1,  1997,  the  MVR  supervisor  of  a  caseload  shall  audit  for  each 
caseload:  6  active  cases,  9  closed  cases,  3  stams  08  closures,  3  stams  28  closures,  2  stams  26 
closures,  and  1  other  active  case.    The  MVR  supervisor  shall  conduct  specific  performance 
audits  on  all  cases  that  exceed  60  days  prior  to  certification  of  eligibility,  and  the  MVR 
supervisor  shall  conduct  specific  performance  audits  for  all  cases  in  which  the  initial 
Individualized  Written  Rehabilitation  Program  is  not  completed  prior  to  6  months. 

We  will  also  build  eligibility  threshold  training  into  the  format  of  our  annual  bi-district 
training  so  that  each  counselor  receives  training  in  this  area  during  the  year. 

Recommendation  #2 

We  recommend  MVR  Program  management  provide  staff  with  information  to  help  ensure 

compliance  with  the  federal  60-day  time  frame  for  eligibility. 

MVR  Response:  MVR  has  already  changed  its  policy  manual  and  we  will  conduct  statewide, 
an  extensive  review  of  cases  to  establish  a  comprehensive  baseline  of  performance  regarding 
the  60  day  determination  of  eligibility.   We  anticipate  beginning  training  with  our  first  bi- 
district  meeting  to  insure  compliance  with  this  regulation.    Vocational  Rehabilitation 
regulations  have  just  been  rewritten  which  necessitated  a  slight  change  in  our  policy  manual. 
Any  eligibility  determination  that  goes  beyond  60  days  must  have  the  concurrence  of  the 
consumer  and  a  target  date  for  eligibility  determination  must  be  established. 

Recommendation  #3 

We  recommend  MVR  Program  management  establish  and  implement  time  lines  for 
development  of  IWRPs. 

MVR  Response:  MVR  has  already  implemented  within  policy  the  6  month  benchmark  for 
the  development  of  Individualized  Written  Rehabilitation  Programs  from  the  date  of 
application.    This  critical  juncture  of  6  months,  combined  with  the  requirement  that  all  post  6 
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month  cases  undergo  a  performance  review,  should  address  the  concerns  of  the  auditors  and 
at  the  same  time  preserve  the  flexibility  within  the  program. 

Recommendation  #4 

We  recommend  MVR  Program  management  ensure  counselors: 

A.  Receive  appropriate  training  for  development  of  measurable  requirements. 

B.  Develop  measurable  requirements  by  reviewing  IWRPs. 

MVR  Response:  We  agree.    This  is  a  difficult  subject  which  will  require  continuous  training 
and  we  will  build  it  in  to  our  annual  bi-district  training  sessions.    MVR  has  already  revised 
both  the  IWRP  and  the  case  review  instrument  to  focus  the  counselor  more  accurately  in 
terms  of  developing  "measurable"  performance  criteria  specifically  for  the  purpose  of 
evaluating  the  IWRP  strategy  and  goal.    In  addition,  the  Reauthorization  of  the  Rehabilitation 
Act  is  happening  this  fiscal  year.    It  looks  as  though  we  will  need  to  change  policy  again  as 
the  IWRP  will  be  replaced  with  a  new  plan  called  the  IPE  (Individualized  Plan  for 
Employment).   Provisions  of  this  plan  will  take  place  simultaneously  with  the  passage  of  this 
legislation. 

Recommendation  #5 

We  recommend  MVR  Program  management  clarify  policy  to  help  ensure  counselors  consider 
an  individual's  strengths,  abilities,  and  capabilities  when  planning  his/her  vocational  future. 

MVR  Response:  MVR  has  currently  published  a  policy  on  this  topic  to  provide  guidance  to 
counselors.    The  policy  has  been  approved  by  the  Montana  Vocational  Rehabilitation 
Advisory  Council  and  will  also  need  to  be  approved  in  our  federal  state  plan.    It  reads  as 
follows:  MVR  will  provide  all  individuals  participating  in  agency  services  with  the  means 
and  opportunity  to  exercise  informed  choice  among  suitable  vocational  goals  in  terms  of  the 
individual's  unique  strengths,  resources,  priorities,  concerns,  abilities,  capabilities,  interests, 
and  needs.   This  choice  also  applies  to  services,  service  strategies,  and  qualified, 
competitive,  providers  of  service.    Individuals  will  be  provided  information  and  orientation  at 
application  and  again  at  the  development  of  the  Individualized  Written  Rehabilitation 
Program  regarding  the  availability  and  scope  of  informed  choice.   Information  regarding 
services  and  providers  shall  include  at  a  minimum:  cost  of  service,  accessibility,  duration  of 
services,  qualifications  and  range  of  expertise  of  providers,  ability  of  provider  to  provide 
integration,  and,  when  available,  consumer  satisfaction  with  the  providers  of  service. 

The  issue  of  primacy  in  terms  of  client  choice  and  counselor  judgement  is  one  of  continual 
training.    Neither  party  is  by  definition  primary  although  it  is  the  interpretation  of  the 
Commissioner  of  Rehabilitation  Services  that  the  client's  voice  is  the  first  voice.    The 
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counselor,  however,  must  deal  with  the  issues  inherent  in  ability  and  capability. 

Recommendation  ^6 

We  recommend  MVR  Program  management: 

A.  Ensure  counselors  are  completing  and  documenting  annual  review  of  IWRPs. 

B.  Clarify  established  policies  to  define  periodic  review  of  IWRPs. 

MVR  Response:  Annual  reviews  will  be  a  priority  item  in  our  agency  and  we  have 
immediately  begun  steps  to  implement  more  training  in  this  area.    This  will  continue  to  be  a 
priority  case  review  item  until  we  have  evidence  that  the  lack  of  an  annual  review  is  a 
statistically  insignificant  event.    Regarding  the  issue  of  periodic  reviews:  this,  we  contend  as 
a  training  issue  and  we  agree  it  is  a  critical  one.    It  is  linked  to  the  issues  of  measurable 
criteria  and  the  overall  evaluation  of  the  strategy  of  the  Individualized  Written  Rehabilitation 
Program.    This  has  been  identified  as  a  priority  case  review  issue.   We  have  revised  the 
Individualized  Written  Rehabilitation  Program  format  to  lend  itself  more  naturally  to  the 
periodic  review  and  have  begun  training  on  this  concept  in  our  bi-district  training  sessions  of 
1996. 

Recommendation  ^7 

We  recommend  MVR  Program  management: 

A.  Clarify  IWRP  amendment  policies  and  procedures. 

B.  Ensure  counselors  amend  IWRPs  when  necessary. 

MVR  Response:  Again,  we  have  identified  this  as  a  priority  training  issue.    Not  every 
significant  change  in  a  consumer's  life  needs  to  result  in  an  amendment  to  the  IWRP.   Those 
events  may  be  noted  in  the  case  record  and  if  they  point  to  a  need  to  change  the  service 
strategy  then  an  IWRP  amendment  may  be  in  order.    We  have  begun  a  policy  review  and 
rewrite  and  will  train  again  on  IWRP  development.   With  the  changing  language  in  the  1997 
Reauthorization  of  the  Rehabilitation  Act,  we  will  probably  be  rewriting  this  policy  again. 

Recommendation  #8 

We  recorrmiend  MVR  Program  management: 

A.  Emphasize  the  need  for  financial  summaries  and  documentation  of  client  contributions 
through  policy  and  training. 

B.  Review  closed  files  to  monitor  compliance  with  the  policies. 
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MVR  Response: 

A.  We  have  already  revised  our  financial  policy  to  strengthen  our  financial  standard 
practice.    The  following  language  represents  the  changes  in  MVR  policy:  The 
counselor,  in  calculating  the  financial  need  of  an  individual,  will  identify  and  utilize 
all  available  financial  resources  of  the  individual,  the  individual's  spouse,  and,  if  the 
individual  is  unemancipated,  the  financial  resources  of  his/her  parents.    In  calculating 
the  availability  of  similar  benefits  and  MVR  cost  participation,  the  counselor  shall 
require  a  copy  of  the  student  federal  financial  aid  award  letter  in  the  case  record  prior 
to  funding  the  second  quarter  or  semester  of  school.    The  MVR  counselor  shall 
review  the  individual's  financial  status  annually  and  amend  the  financial  report  and 
cost  sharing  arrangements  as  appropriate. 

B.  MVR  will  review  closed  files  as  outlined  above  to  monitor  compliance  with  this 
policy. 

Recommendation  tf9 

We  recommend  MVR  Program  management  establish  controls  to  verify  income  for  a  sample 
of  MVR  clients  on  a  routine  basis. 

MVR  Response:  Thanks  to  the  help  of  the  auditors,  MVR  has  identified  the  possibility  of 
contact  with  the  Department  of  Labor  to  see  if  there  is  a  connection  we  could  establish  to 
randomly  check  income  levels  based  on  case  review  lists  generated  for  the  compliance 
policy.    We  will  begin  immediately  working  with  the  Department  of  Labor  to  share  this 
information  so  that  income  can  be  validated. 

Recommendation  #10 

We  recommend  the  MVR  Program  strengthen  management  controls  by  requiring  an  award 
letter  or  other  necessary  information  prior  to  paying  education  expenses. 

MVR  Response:  The  financial  award  letter  in  the  file  is  now  a  part  of  agency  policy.   The 
only  time  when  this  is  not  possible,  according  to  federal  law,  is  in  the  first  semester  where 
receipt  of  this  letter  would  imduly  delay  services  for  the  individual.   No  training  after  the 
first  quarter  or  semester,  however  can  be  provided  by  MVR  until  the  financial  award  letter  is 
received. 

Recommendation  #11 

We  recommend  MVR  Program  management: 

A.        Establish  a  policy  to  review  cost  information  including  direct  and  indirect  costs  for  all 
providers  and  services. 
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B.         Develop  a  process  to  routinely  review  fee  limits  to  establish  reasonable  fees  and 
ensure  compliance  with  0MB  Circulars  A-87  and  A-122  and  ADA  requirements. 

MVR  Response:  MVR's  Community  Rehabilitation  Specialist  has  already  begun  work  with 
DPHHS  auditors  on  this  subject.    We  are  currently  awaiting  the  result  of  a  state  wide 
community  rehabilitation  program  audit  regarding  fees  and  0MB  compliance.    The  agency 
will  then  re-review  its  community  rehabilitation  program  rates  and  the  process  for 
establishing  them. 

Recommendation  #12 

We  recommend  MVR  Program  management  identify  and  generate  information  to  help 

manage  program  resources  including  budgeting  and  analysis  of  purchasing  in  each  region. 

MVR  Response:  MVR  is  currently  in  the  process  of  completely  revamping  its  computer/data 
base  system.   Budgeting  and  analysis  of  purchasing  in  each  region  are  both  specific  reports 
that  we  will  be  generating  in  addition  to  a  lot  of  other  information  which  should  help  the 
program,  particularly  with  its  evaluation  component.    We  intend  to  work  with  the  Discovery 
2000  program  to  emulate  the  types  of  reports  recommended  in  this  section.    As  we  increase 
our  skill  in  gathering  the  information  in  this  section,  our  capability  to  manage  detailed 
district  information  will  increase  dramatically.    In  addition,  MVR  is  looking  at  the  possibility 
of  even  more  specific  regional  budgets  in  an  effort  to  allow  our  regional  managers  greater 
flexibility  and  control  in  managing  their  budgets. 

Recommendation  #13 

We  recommend  MVR  Program  management: 

A.  Revise  goals  that  relate  more  to  the  purpose  and  mission  of  the  program. 

B.  Establish  measurable  objectives  which  relate  directly  to  goals  and  allow  for  attainable 
results  or  outcomes. 

MVR  Response:  MVR,  through  its  state  and  strategic  plans,  is  currently  working  on  an 
agency  wide  planning  process  to  review  goal  setting  and  performance  appraisals.    The  federal 
Rehabilitation  Services  Commission  is  currently  working  on  a  series  of  bench  mark  indicators 
that  the  states  will  be  required  to  use  in  measuring  outcomes.    In  addition,  the  department 
itself  is  working  on  a  series  of  bench  marks  that  allow  for  identification  of  setting  outcome 
based  goals  and  then  measuring  them  accordingly.    These  federal  and  state  bench  marking 
systems  are  expected  to  begin  during  the  next  federal  fiscal  year. 

That  said,  however,  the  MVR  Strategic  Plan  and  our  cycle  of  service  contributes  a  great  deal 
to  the  concerns  of  having  more  tangible  measurable  goals  with  objectives  governing  the 
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agency  at  the  broadest  level.   MVR  is  an  agency  that  is  easily  measured  because  of  the 
outcomes  that  we  and  our  consumers  achieve.    The  five  major  goals  identified  in  the 
Strategic  Plan  and  the  basic  production  requirements  of  the  program  will  together  begin  to 
affectively  address  these  recommendations. 

Recommendation  H4 

We  recommend  MVR  Program  management: 

A.  Identify  what  information  would  be  useful  to  help  manage  program  operations  and 
measure  program  outcomes. 

B.  Develop  that  information  into  useful  management  reports. 

MVR  Response:  MVR  continues  to  move  toward  an  agency  wide  computer  system  which 
will  enhance  our  means  of  achieving  this  recommendation.    We  have  also  connected  with  the 
Montana  State  University-Billings  Rehabilitation  Counseling  faculty  and  we  are  currently 
having  discussions  regarding  the  basis  for  evaluating  the  effectiveness  of  our  program.    We 
are  currently  working  with  staff  at  all  levels  to  identify  which  information  would  be  the  most 
important  to  them  to  help  manage  our  program's  operations  and  our  outcomes.    Our  new 
computer  system  is  helping  us  tremendously  to  identify  more  and  better  information  that  will 
enhance  our  ability  to  do  a  better  overall  job  of  program  evaluation. 

Recommendation  #15 

We  recommend  MVR  Program  management  develop  performance  measures  to  help  assess 

the  success  and/or  effectiveness  of  the  program,  both  at  the  regional  and  overall  state  level. 

MVR  Response:  MVR  is  currently  developing  performance  measures  which  include  case 
management  as  well  as  rehabilitation  outcomes  so  that  we  can  better  assess  the  success  and 
effectiveness  of  the  program.    In  addition,  as  was  previously  mentioned,  the  Rehabilitation 
Services  Administration  is  developing  a  series  of  eight  standards  and  indicators  by  which 
states  can  evaluate  their  own  individual  performance.    States  that  fall  below  certain 
performance  levels  will  be  required  to  implement  corrective  action  procedures  in  conjunction 
with  their  partners  in  the  federal  government.    We  hope  to  have  our  standards  and  indicators 
in  place  during  the  next  federal  fiscal  year  which  begins  October  1,  1997. 
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